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STSSSTSSsflp fftlSSsflsi SyQJjLDLD 3ilSSsfl [58iSlSSI 


srsmsmt G 1 sumb srssrug] £g )(/£<950 " 

srssrp Sil(Tijd(&jfD6i?l6sr uudeep learning uprfil g ?0 L/pp3pss)p srQp^lsSlueosLCi srssrp] ^jsssflrs^] 
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1 Deep Learning 


^)tu/5^)/T suL^ld &rnrn<sfi<ssT (MachineLearning) 6£0 u^^nurrs, rffu^rrsb QfBilQsumjdsro stsstu^] 
^SS)LlHLjli). <J)/<5/751/0/ LD6sfl&6S)l6S)l_lU QP<SS)GtT GTSIIGU ITpi SpSilpgl <oTg£tU 6S)&) Qft6Sr(l>6STrTlSI-UJrTad Qamsssr® 
a_ 0 su rrdauuill— Qp rfjlu^rjsi) QpiLQeorrijdGfo ^(pil.QppsSIsb s ^0 (^Lprsemp i3pd(^LbQurrpi <9ips$)iG$)i—UJ 
ZLpGS)<snd(3j GpesrpiQiA Qpfflujrrgj. &Lfhu£§l<sS)(njp§i ^rnLiSl^g] iSlssrssrrj spsijQsurrq^ aS\<sq,uJi£>n&d 
ajD§i)jD&il- <jy3>/7<si/0/ (3jLprB<ss)&iu5)6$T pLD&nmuSlsoimm <spm ^penm ^rriLLif^lg^rrmssr) 90 li^Iuj 
aSlG^iupempd apdlpgi. ^(Kppprra LcpQprrQjj /s^ldl/ (mpQprr(p r£lu^rjnesi)6jpQaeurGsu appjd 
QarrsmQtsfTtsn sdlGi^ujpQpfTf^l G&rjpgi ^lssrQssm(r^ Lf^nu GfilGtyiupeinpu-iLb appid QamsrrSlppi. ^jdisanCSp 
pnpi L^sbeShuesT STemressfldenaulsb £g) 0 <* 0 LO usbCSeopj QpGsxsfTpijLDLiaGrT GpGsrQprrQi—rTGSTpj GussxsoulihssTGSTGb 
eoupedlsb dHanassrdauuiLd) (rglu^rjeb QpiLQsorrijdGro) Qprri—ijddUurTa npiunpi sfilGtyUJrhiaGsxstTd appid 
QamsssrQL- Gn^Slesrpesr. ^ispuuGSU—turrai esieuppi ^(pGimdauuiLi—Gp QpLLQsorTijdGiiv 

STGSlIll ppglGUll ^,0LD. 


(g£)d>(5d>3jl<su ld nssT&ji ilsirsuQ^il snLDUilsir ^Lpsoil Q&ujfpsn&iUJrT&i r5LD§] asssflssflulsb 

^(TifGijrTdsiuuQdltngi. ^GuinnSlsb (Lpgsbrflemso, ^sni—fisnso, assu_r§)emso srepiil 3 
rglsmsoasiflsb usbGsu^j rglg^rrirasrash ^snLDdauuQdlssTjDSST. asmddQaissnjdarrau 
uuj<ssTuQ)d,^,ijuQ)Lb (Lp^ssrenm ^gv(&, rUlu^rjmssT STesrrru <3ienLpds,uu(Kul. gasi/(o)si//70 
rfilu^rrmspul Qsudi—ij erspiil <^isodssr pldgviI prhi&qsrijd&rTSST ppsyaenGrru (olufDtgjd 
Q&rrsrrdlsiTinssT. (Lppsb rflsnsoulsv ^shen rffu^rrrTsir&srr asrrsifLLQd&rTSSTgi. &>s$)L_ rffsnsoulsv 
2 -GfTsn fiu^rjirm&sh (olsueiflul lL® da assign, ^sni—ul so spssrrrn ^soscajj <3i^JD(g) GlcjdulIl. 
L£>ss)(D(Lp& (j(id!d)si) usbGsunpj rffg^passrassisn ^(j^surrdd, spsijQsurrq^ <*uCh)d&>)soiil ^Lod^ffluj 
<SHSfTsy(rijd&is$)SfT tBrjsyaqsnfisZr ^snsssrpgi asmddQ&smsrrg Gl^m^rd^dlssrpssr. <jy<5/7si/0/ 
uiilfodddanssr ^rrsii&onmu Quip fry silmrd^soG^ (Lppsbrflsnso ^erreifil(K Qsudi—ij ^(gjil. 

£g)si/ QGild /_0/_s37- weights LDtogjil bias-god G&rjpg] ^ssu—iil ensoul sir (Lppsb <^(K)dd)sb s^shsrr 
rgJu^ijrTSSTash gssr&ii aemd!d(K)a6S)sn£ QgrTi—ihit&jdlssTfDSST. ^]iliD^)uLisi(Sf^i_ssr ilemQnl 
weights LDipQjil bias-god G&rjp&j ^sni_r£)ensoulsb <3i®^ < 3 i®d(&j ansssruuupssr, <3igJ Resign 
asssrdd(K)aenGn£ QgrTi_[hi(gjil. £g )elenaGfo <3i(K(b&>(K(b&> rfiensoasiflsb asssrd&Qlash 
rflaLpppuuLl® aeni—d rglensoulsb s^shsn r£) u^pnesr gesiggprediction-go QsusrflulddljD^]. 
^jll(LpeniDulGsoGuJ rilg^ijsb QpiLGeorrld posTdamssr Luflpesxsou-iLl appGsxsoujLC) (SLopQamsrrdppi. 
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Deep Learning gtgstu^i )/£&> rglu^rjsb QrBL-Qsnmjsts)<3iu uscQgii^j sfilpihjasiflsb 
^pmu&iljDg]- Simple Neural Network, Shallow Neural Network, Deep Neural Network, Convolutional 
Neural Network, Recurrent Neural Network Qumssr/o usbQsu^j sfilprhiaGiflsb ^j<sufo svtjD ^-(r^Gurrddl, 
pfjGiiaesxsrT <3160&) ^pmurs^] si/DQjd GlamsndljD&i]. ^jsa/f)SS)/D 2-(ri)Gu rrdq^Gu^fBQ&QST Tensor Flow, 
PyTorch, Sonnet, Keras, Gluon, Swift, Chainer, DL 4 J, ONNX QurresrjD ugoQgu^j aifisimDULi&m 
deep learning-sv 2 _mm< 5 $r. ^su/D/flsv GyssTfonssr Tensor Flow gul^KSuj deep learning-sb 2_m<srr 
ugoQguqj ^GmmjLi&GsvsrT it, mb 2_QTjGijrTddld <£/D<s6U/T/i>. ^j^foQ' 9 i<ssT (Lp^sflav TensorFlow-sb 
2 _<shm ^]LpuuGis)i_ujrTGisT siilGtyUJrd&GtixsfTU u/pifld &if)rr)]dQ8in<shQ<siimh. 
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2 TensorFlow 


Deep Neural Network-sir usvGsufrp Q&iusburrQasnGrr ^rrmusu^fd^ 2 _<$ 6 iyzi> susnauSlsb 2011-sb 
Google brain (a,(Lpsurrsb 2_06i//735<sl/l/l_z_ spssrQro Tensorflow £g)0y LSlasyih aipssrLDrrssr 

<35633 fig Gl&iusburTQasinm &souLDrTd(3jsiJgfD(aj sjjdjd susnauSlsb 2_061//735<sl/l/l_l_ dhrofa, ^Lpso 
<35633 flp library ^,0/A ^j^ssr ^Lpsoih <$/76iy<356)T ufDffihu ^Lp ld rrssr l jfl^smso rpmh GLDfdQarrshsn 
(Lp UyLLj LD. 

<5/761/35 err 6775<$ ^sdlduiSIsv G&LSlpgj smsudauut^SlsirrDQ^rT data model-^ssr&ji 

Qissr&rrij srssrrpj < 3 iss)Lpdauu(f)i-b (Data Model = Tensors). ^j^Qurrssrp usbGsurpj 

Gh _ ssranfa^r^ddlssu _ uflsb <$/76iy356i t u ftumjDUULL® Gl&iusou rr(f) 356iT r£}&>ip6uss)8jGuj Tensorflow 

STS&riQj ^ss)LpddlQfDrTLCi ( Data flow = Tensor flow), (fff^snaiu uflumjDjDrhJ&snsnd arrsm 2_<g6)y/i> 
< 3506 i 5 )( 3 uj Tensorboard ^ 0 /i) (Dashboard = Tensor board). ^d&QTjGrShfilsb usbGsuipj 

Gil 6OT<97777<3>(6y5<9>® 63)Z llSlsO <g/76)y<S6)T STSUSlfl^LD U ffiLD /77D UU® S)SSTJD 63T STSSTU 6£0 61/63) /JL// LD 

Opsoil) suss)rjrf§i amli—UU^Slp^i. (gfi&jGsu Tensor flow graph srssrrru ^ss)Lpdauu(j})LCi. 
£g)6i/6iv63>/7L//_/i> nodes ld/dpjld edges srspnh (ffrrsm® ^Lbardasnsrru GlujDrfilQTjdt&jLD. Nodes 
STSsrug] 35633 Gl&iusbu /T(/j)3563)6)ray i£> edges stsstu^] Qi_ssrarrss)piLi ld (&jrSld(&jLb (Data flow graph 
= Tensor flow graph). (ff§I CPU, GPU, Server, Mobile Gurrssrro aQ^stfasiflsv rfJ^jsi^su^jD^ sjroro 
SU 63)35 u5)6U &SOULDfTSST 35/_l/_63)i£>/j/_5)63)63T/j Q/_//Dfl5)035(3>L£>. 


S-3jmressT rflireb: 

“hello world” srspuh surrddlmfss)^ QeiJGifluuQpgiGiJgfD&rTSST 2 -grrpsmLb iSlssTsuq^LDrrp] 
<31SS)LDlLjLD. £g)<$63T £LpSOLD 6£0 LDIT^lfl tensorflow f) [r®S] SSL <3] STUD U L] (jp GS)fD 6S) UJ £ 
Garrsnsnsorrif. Quag] surra 3 rilss)soass)srr ^sfrsrri^ddluj^]. 

i) Gh-.Gsr&rrrjaGiflsir a-QTjSurrdaLD - Construction 

ii) Ql_ 657377773563)6)T ^jiuddl 35 633735 65(4) 35 65) eyr r£}&Lf>£aj]gsb - Running a session 
in) (LpipsLf&>s$)6fT <SH,rrmu^sv - Routing & Analysis 

https://gist, github. com/nithyadurai8 7/8c2d24e08cebd 7dcff4696727c758913 

import tensorflow as tf 
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a = tf.constant("hello world!") 
print (a) 

s = tf.Session() 
print (s.run(a)) 
s.close() 

with tf.Session() as s: 
print(s.run(a)) 


rflrjspid&msisT etieird&if: 

1. (Lpgstisb tensor flow library-go tf srstpiif Quiuflsb ^)/D<S 0 ro^) Q&iugjiQamshsn GsusmQif. 

2. iSlssrstsrf library-d(&,<sh sshsn ^ssxssr^gi operators-go iq if tf srssrd 

UUJSSTUGsvsmQif. £g)/5V0 "hello world" <srss)iif constant-go QsuGifluuQf^ 
stiQTjifiqdlGfDmf. (oTsstGsu tfconstant() ^rsstu^fq^sh ^£idQ&rT(DQ(Dm—!j Q&rrQd&uuLL® a 
srssiiif Qi—Gsr&rrij ^(j^<simd3iuu(f\Sl(D§i. print (a) sr&srd QarT@d(&jifGuiT§i ^§1 constant svssia 
Qissr&rrssiij QarrsmQshsrT&n (oTgstlkss ><$ ldlLQGld QsuGifluuQgajjif. ^Uftstpismi—UJ 
l£>$51ul51<5S)gst QsvstiuuQggrT&j. £g)g/G >so (Lpgsb rflssisviuassr Qi—tssr&rrrjasiflssTa-QjjGijrTdaif 

3. <3i(fid,&> rflssisoitisb^rrssr Qi—ssr&frsmrj ^luddl iD^uiq&ismsrT QsusifluuQf^u GurrdlGjDmf. 

£i—{5 si/ <sijQ j 3) sessionQ GTSsyif operator ^(ajif. $l<sv>& itisstso 0 if ^jpsm(f) stigihi&stisb 
£g) ludasomf. 

sti^ifl: tfsession() sr&srd Qs,rr(f)f§j s srsg) i if Qisir&rTsinp ^(r^surrd&isqif. iSlssr&srijs.runQ (orssrd 
Q&rrQggi constant-god Q&rnsmf^mm Qi_ssr&[TGi n/r ^tuddl LD^luitississrd amsmsonif.. print 
(s.run(a)) stsstliG^ constant Q&rTsmQshsrT w^uitistossr Qsostiuu(f)f^]if. asso—dhurra 

Qi _ ssr^rrf&iss>srr ^jiudd) (Lpisp^p iSlssrssrrj prof a-QTjsorrddliqsrrsrT session-go ril^j^^l stii— 

GsosmQif s.close(). «£0 stigif. 

stilus if 2: with tf.Session() as s: srssrd Q&rr(f)£§j ^jS/rjgsfr prof ^jiudsi Gsosmupiu 
^ssissrfsto^iLjif Q&,rr(fida,<sorrif. u)/dQ/d/t 0 stigif. ^jif(Lpsts)/Du5lsb rsrrif ^jiud^ifGurr^i 
assu—dhuas, session-go rflirpf^f, Ggstnsoitisbstnso. 


rihjGpid&rresT QsostiufCK): 


14 





Tensor("Const:0", shape=(), dtype=string) 


b'hello world!' 
b'hello world!' 

2.1 Constants 

90 _ rflGsxsoiumssT io^iui 3 ss)Qnu Qurb^i ^iiudi^so^rb^ tf.constant() otct/ld 

operator uiusiru($iSirD§i. string, int, float, bool (Sumssrp usbQsa^j Guenaaetflsb pjjGyaGsxsrru Quid^i 

£|)(L//57(3jL0 dgsi tss)ld astM-iugi. dLfidaesisrL- OT(J)d55i/<55<5/rL!_iij_®L> £g)dg sir usbQso^] tzrjsg susnaasnmd arrsmsomb 

https://gist, sithub. com/nithyadurai8 7/d 7ede4 7ace 7ce9028342f8153b81c9b3 


import tensorflow as tf 

print (tf.constant("hello world!")) 

print (tf.constant(1)) 

print (tf.constant(l, tf.intl6)) 

print (tf.constant(1.25)) 

print (tf.constant([l,2,3])) 

print (tf.constant([[1,2,3],[4,5,6]])) 

print (tf.constant([[[1,2,3],[4,5,6]], 

[[7,8,9],[10,11,12]], 
[[13,14,15],[16,17,18]]])) 
print (tf.constant([True, True, False])) 
print (tf.zeros(10)) 
print (tf.ones([10, 10])) 


t,ffijeoidarrasr aSlsfT&aih & QeudiUifCh): 

1. “ hello world” sresid QarTQd^LtGurTgi, tSdgj string sussra Qisir&manrr P_(r^surrddliLjshsnss)^d 
Gmsmsomb. dtype toTssrugi data type-god (^/rfld^LD. 

Tensor(''Const:0'', shape=(), dtype=string) 
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2. erem 1 eresrd G)3irr®d(^LtiGurr&j, int32 eves>a Qi^esr^rresip s^q^evrrd^LC). £g)g/<3<si/ default grjey 
suistna ^(3)il>. ^jest^Giu intl6 eresr LcrrjDjD aSlqrjLbiSlesmeb, 1 eresrd Q&rrQd^LCiGurTGg 
uda£®eb, inti6 eresr ^eifld&i (Severn® ld. 

Tensor("Const_l:0", shape=(), dtype=int32) 

Tensor("Const_2:0", shape=(), dtype=intl6) 


3. &j&LD erem&esimd Q&rr®d(&jLbGurT§i 1.25, float evesia Qt^esr^rrestp ^(r^evrrd^ih. 
Tensor("Const_3:0", shape=(), dtype=float32) 


4. [1,2,3] eresnid list LD®uiSles>esrd QarT®d(3)Lt>(SurT§], <s 4 ges>esr s ^0 urflumem array-^ad 
aessrddleb Q&rreh(erj)il>. ^pesr shape eresinl) uemiSlesr ld®u l/ arreShurra £g) ebeomceb, 3 eresr 
£g)0L7L/63><$<s arressreomd. ^prrevgj Id array-eb 3 columns s^ehenest^ ^§1 (gjjfildSljDgtf. 

Tensor("Const_4:0", shape=(3,), dtype=int32) 


5. [[1,2, 3], [4,5,6]] eresnid £g)0 urjlihrrem array- ®od Gl&rr®d(&jid(Surrgi, shape uessnSlesr LD®uq 
(2,3) eresr ^errenest^d &,rremeorrid. ^prrev&j G)arr®daiuuLlL -_ 2d array-eb 2 rows mro^jid 
epevQevrrq^ row-etflepiid 3 columns 2 _erren§i eresrues>g £g)0/ (^nflddlfo^]. 

Tensor("Const 5:0" , shape=(2, 3), dtype=int32) 


6. [[[1,2,3], [4,5,6]], 

[[7,8,9], [10,11,12]], 

[[13,14,15],[16,17,18]]] eresnid (LpuufilLbrressr array-god Qs,rr®d(^ ) idGurrg > i, shape- esr ld^Iul/ 
(3,2,3) eresr ^ehenest^d arressreornd. (Lpgedleb 2_errerr3 eresrug] opesrigj 2d array-®od 
Q&rressr® efrerrgi eresruest^d (^(Sld®rD§i. rS8®^§l £-err err 2,3 eresrugi ^ggesi&uj 2d array-eb 2 
rows, 3 columns 2 _errerrgji eresruest^d (^rjSldSljD^i. 
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Tensor("Const_6:0", shape=(3, 2, 3), dtype=int32) 

7. True, False <oT6sru§] Gumssrp LD^ULi&smmd G)arr®d(3jLC)GurTaj] bool gugs)& Qissr&rrsmrr 
S-QTjSiJrrd(&j<5iJ(SS)gd simsGsrsorrih. 

Tensor("Const_7:0", shape=(3,), dtype=bool) 


8. tf.zeros LdippjLb tf.ones istsstu^i (jpanroGuj L/GogShuLD ldlLQll ldjdqjld 1-gg ldlTQld Quip fry 
sSJmrhi^LD ^(r^Gurrd&u uiueiru(^dlp§]. ^$rD(&j<sb Q&rrQd&iuuL-QshsfT 

LD^ULi&issxsfTU Qurrpjp&j ^jGS)GU (LpenjoGuj 10 columns -go lclLQld Quppj aS\<sfTrh](&,Lt> Id array- 
gou-iLCi, 10 rows & 10 columns -gou Q up try aSlsmhJ^iL 2d array-god Q&rrsmu Gluesr&msnrj 
^(r^surrddliLjmsn^i. 

Tensor("zeros:0", shape=(10,), dtype=float32) 

Tensor("ones:0", shape=(10, 10), dtype=float32) 


2.2 Tensor properties 

6£0 QlSST&mj <oT6STU3j] QuUJlj, GULCj-SlJLD, ^/TSiySl/S5>d5 <5T<57S)ILD 3 U633TL/<S63>fflT.® 

Glarnsm® aS) Gerdau u@§i)fD&ii. 

Quiuif (Name): <$(§ Qusir&rTGvrr s^q^simd^LbGurrGp, [BL£id(^ Geuemi^iu Quiuss)pd Gl&rrQpg] 
S-Q^Gurrd&Gcrrii). ^j<sb<ss)GoQuj<ssfl(sb, Grrsp operator-gou uiussruQpS)iLj<shG<annGLOrr^psir 
QuiuGrr Qwssr&rriflGsr Quujprrs, prresm <s ^t<ss)LD^§i<siSl(^Lb. ^disunGp spGp Quiusnij lSssst^lL 
lSssst^ild uujGSTuQp&jiLbGurrGprT, <S8<SV60§] Quiuffli—rrprT tspGpoperator-go lSgsst^iL lSssst^ld 
uujissTuQpajjLCiGurTGprT, ^pssr udapSilsb underscore (J (LpenjoGuj 1,2,3, gtsst sufies)# 

<oT<s$sr&6fT Qsu<sifluu(^(su(ss)pd amsmsorTLC). 
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su Uf.su ld (Shape): £g) uussstl) Qunajisung; shape=() ld^Iuss)uQuj QurofSlq^d^iL. ^ssrnsb 
<3IGStiflaGS)snd (g/ifuiSl (fid flung] uuL/fid, 3]g] sy(g urflumsmum, ^(njurflumsmum ^sbsog] 
(Lpuu flLDirsssrLDiT srsstustgd (gjrfldau uiusstu/fih. difdaismu 2 _gnrrsmg&>lsv 
^uusmiSlstsstu uiussiu(fg 5 ), sp(g Qussrnncgstuiu suuj.su ^stmustud Qg;nsmu 
LU/f)Q/D[T(g Qissr&rrij S-QTjSurrdaiuuLLQsrrsrTgji. 3]gnsugi cl srstiil) Qissr&rrjfleb 2_shsng] 
QunsoQsu 3 rows-god Glarrsmi— 1 -go lougfif Qujdq] sSisnfifgiL e <3isssfl £g)/&/0 
s-(gsund^uuil(fsnsngi. tf.ones() srssrugi 1-go ldlLQll Quit) fry sfilsnuiiggLU &§jrj ^sssflss)iu 
^(gsund(gid.. gtsstGsu d. shape [ 0 ] stsst 3 rows-go ldlLQll (grfydtgLD sftgLunaid Qg;n(fggnsb 
flung] Lansing], ^gjofaiiDU columns-go S-Qjjsundd) &gnj ^sssflstiu 2-Qgsunddld QainshisgLU. 

grjsifsust& (data type): int, float, string, bool QunssrjD usbQsu^j grrsij susmaastflsb sy(g Qissr&rrij 
snfgg grrsij sustaismud anfgggi srsnustsigd (gnSlda; dtype srssrugj uiusnu/fdlrog], syrg grrsij 
sussiaissnu LD(oQ(Dn(nj grjsij sustaining; Lunfonysugrotg tf.cast() srstud operator uiusnutfidlgg], 
dipdarsmu _ 2_gnrrsmg&>lsv float srsnug] inl- 3 J,a runjDguuLL/fshsng]. 


https://gist.github.com/nithyadurai87/70cafef972154224c891220aec2336c3 


import tensorflow as tf 

a = tf.constant(l) 

b = tf.constant(2) 

c = tf.constant(l, name = "value") 

print (a.name) 

print (b.name) 

print (c.name) 

d = tf.constant([[l,2],[3,4], [5,6]]) 

e = tf.ones(d.shape[0]) 

print (a.get_shape()) 

print (d.get_shape()) 

print (e.get_shape()) 

c = tf.constant(1.25) 
print (c.dtype) 

print (tf.cast(c, dtype=tf.int32).dtype) 


r&ijspidanssr Qeu&fUtffit): 
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Const.O 


Const 1:0 
value :0 
0 

(3,2) 

(3,) 

<dtype: 'float32'> 
<dtype: 'int32’> 


2.3 Operators 

Qistr&rrrflsb uebQsu^j ^uuQpiCi^pash ^shsnssr. ^GUjDnfleb disc surras sprf srsmsssflsir surjds, 
Op sold amssni&sb. £g)(5 ^sssflsisiflsir a^Ldi^sb, Quqtjd&sb Gurr<sir/DSiJ/Dji9jD(^u uujsstu^ld 
^uuGrjLLi— rj&srr dGip Q&rrQd&uuLlQmmQST. 

https://sist.sithub.com/nithvadurai87/811fb2f2d2871420f097c5c99c659f9b 


import tensorflow as tf 

a = tf.constant(l2l.5) 
b = tf.constant([[l,2],[3,4], [5,6]]) 
c = tf.constant([[7,8],[9,10], [11,12]]) 

rl = tf.sqrt(a) 
r2 = tf.add(b,c) 
r3 = tf.multiply(b, c) 

print (s.run(rl)) 
print (s.run(r2)) 
print (s.run(r3)) 


rjhjGpid&mssT QeuefUidCiSl: 
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11.022704 


[[810] 

[12 14] 

[1618]] 

[[716] 

[2740] 

[55 72]] 

2.4 Variables 

6^0 variable stsst sus&ipujrgjdsiuuL-L- Qissr&rriflssT LDSiluLi&GsvsrT rBLDiurrsv umiD/fil LDrT(D$ 
^ssupdai (Lpuj-U-i ld. fJ^psb QtblIQsu rrfddlsb weights, bias Guitsstid <=gy errsy 0 dass)err pm!* 

LcrTfonSl LDrrjDnSl ^ssuoda Gsu6ssriq.u5l(i^d(^LCi. £g)g/ GurrGsrrr) g^i^rhi&siflsb £g)/5<5 variable-go t^ttip 
uiusiruQgSild Q&imsrrmsvml). 

dipd&tsmi— a^prrrjsmptslsb, a Ground variable S-(rij6urTdauuLL(j])shsn&ii. g^gsussr 
a_0siy/T<®0/i)(3/_//T^/ ^gssr Quiurj (vl) LLjngpid [3,3] suiq.suid Q&rTQd&uuLlQGrrsrTg]. ^i^nsu^] 
3 rows LDjDiryLD 3 columns-god Q<9 uts$stl_ <sp(rq ^sssfl ^(f^surrds.uuLd^shsn^].. spQjj session-go 
S—QTjSunddl <sq§)isb a-gou iSlrflsmid Q&iugj urrrtggrrso, random-^(r^d^id srsmasusnu 
Qujdjd tSqsssfl GlsusifluuQsusngd arrsmsomi). G id spud, tfglobal_variables_initializer() -go 
£j])ujd(gjsugfn(&j (LfisSTSsrij iflflGmid Q&iiigi urrijf^rrsv, "Attempting to use uninitialized value vl" 
stss)ild <$6i iigj QsusifluuQsusmpd auTsssrsvmd. ^aGsu £g)/5<$ function-go ^ujd(^LdGurT§]^rTsiT 
Q&n(i)id3iijuLLi—variables-r$i6S)6$T£§iih <sqgssT gjsuda LD^Iuurreb initialize Q&iluuuuQip. snfs, 
6£0 variable-go uq id uuJGsru(f)f§jGU^jD(^ (Lpssrssrrj £g)/ 5 <$ function-go ^]iud(^su§] ifls, lS\& 
(LpddllULD. 

b Ground variable S-QijGund&iuULLQshGfTgi. ^j^gsigst ^(TijGurrd^idGurrG^ initializer 
^peoid [5& variable-sb gtsstsst iptfluLf&sh [gfiqTjd&i GsusmQid gtgst GrBpiq-iLmsid Q&rrQgg] 

s£}LLGl_mh. ^ <5/761/0/ GTGSTGST GULq_G>jf&>lGV £g)0<S<S G}GUGSST(f)LL STGSrd <*l71/61/<5/D0 U^lG0fT<3i, 

<311515 GssflGS) ujG id GrBpiq.iurT3id ©<s/7(J)<53i/ gSIlLGi—Ttip. ^jpsussr iSlflsssnf Q&iugi urrijpprTsb, 

<3if>&> tS4GSsflllf GST LP^IULj Q GU Glfl U U (f) GU GS)^d <3>/763376U/7L0. 
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s.opL^ffhurrs, x <oTss)ilL variable 1 OTgo/zi) gjsuds, ld§>1u6$)uu Qujdqi S-QTjSijrTdauuLlQGhmaii. 
lSIsstgst ijsession-d(3j(snfor loop opsoih 10 arfDfrp&GsvsfT ^(^susdSl, spdjQtsimqij &nr)p9<spiLC> <3ir5£ 
variable, 2 OTgn/fi) cons tan t-rM,sb QuQTjd&uuQSljnSfi. ^sbsurrfQj Ql/ 0 <s<suul_(J) £le$)i_£g 
ld^IuQu, s^si/Qsi//T 0 &rnfi5l<SH $ifgj$ 5 lifil<s£iil>, x-sst ld^Iuuits 5 tf.assignO QpsvLb 
^GS)L£hdauu®&)fD&i]. loTGsrQsu^mssr Ghsusifhtf® 1 *2=2, 2*2=4, 4*2=8, 8*2=16 ... <stw sarB^jmm^]. 


https://sist.sithub.com/nithvadurai87/a2bl6555bbe599d089969bf43318265e 


import tensorflow as tf 


a = tf.get_variable("vl", 

[3,3]) 

b = tf.get_variable("v2", 

initializer=tf.constant([[3, 3],[4, 4]])) 

x = tf.Variable(l) 


y = tf.constant(2) 


r = tf.assign(x,tf.multiply(x,y)) 

with tf.Session() as s: 


#print(s.run(a)) 


s.run(tf.global_variables_initializer()) 

print(s.run(a)) 


print(s.run(b)) 


for i in range(10): 


print (s.run(r)) 



rfilrreoid&imssr QqjGifluS’CK): 

[[ 0.71750665 -0.39080024 -0.01946092] 
[0.94270015 0.61512136-0.7205548] 
[0.6800127 -0.81497526 0.7290914]] 
[[3 3] 

[4 4]] 

2 
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4 


8 

16 

32 

64 

128 

256 

512 

1024 


2.5 Placeholders 

Placeholders <oT<sstu<ssxsu ^rrsy&m GurjisfilQTjddhssrrDGST <oTss)iil > (^/fyuonu ldlLQlc) r5L£>d(3j 
s_<smijd > & > i3 uiuGSTuQdUssTfDGST. ^smss)LDUJ[TSST ^rrsij&smm session ^ludi&Zd 
QanTsmi^Q^d^ihQungi run-time s v QuiprpdQairTGrTdUssriDGsr. feeddict CTgy/i) argument QpsoLDrra 
^jssxsu &) /rsi/<s sm sytlj QufD^jdQ^msrrdleiriDssr. Variables <oT6srugfD(3j fSj^rrsuQ^rrq^ a^sads, 

mSiluLi GgGsxsuuuQdlfD&i]. n<5 osxsudt^id, grrssr iSlssrssrij (^)ujrd&£ G)grri—[h](&jLt). ^esrrrsb 

placeholders £g)iL//5V06i/<$/D0 <o775 <$ 6£0 §i<sijd& ld^] uLj ld G^gs)GuuS1gcgs)<so. session ^jiuihjdld 
Q&iremui-(Ti)d(3jLt>GurT§i LD^lui^anssxsnGurr^wmssr^]. dipdansmi— 

Qsu^jid Quiurr LPfDQJih ^/rsi/si/ss>«ss>UJd5 QarrQgg] x, y OTgn/fi) 2 placeholders 
S-(^<sundaiuuLL(^i<sh<snssT. x-gst ld^Iugs)u Gsxsugg] y-esr Ld^lues)ud a<smdd)@6iJgfD&mssT 
aSlS/HLjih Q&in($\d&iLiuLL(j)i<srT<on§i. L5hssT<ssrij ^gjoamssr Lod/luLjaush session ^jiuihjdld 
Q&iT6mL^(iT)d(3jLbGurT§i x-d(&j 100, 200, 300 ldjdqjld random (LpanrouSlsv ^ 4 <ss)LDfB^ 1 Qp^eb 10 
GU6$)[n6}<sorT<ssT GTsm&m umiD/fil LDrrjDffi} <s>i(si?ld&iuu®dl<ssTfn<s$T. ^<sij(ohsurT<ssT^jd(^LDrrGSTy-<ssT 
wSiluLi &<smd§>)i—uuLL® ^ssxsn iSlflessnl Q&iuujuuLL($\<sfT<sfT65r. 

https://sist.sithub.com/nithvadurai87/495e2484a602abc85d3e53ed7ace7407 


import tensorflow as tf 
import numpy as np 
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x = tf.placeholder(tf.float32, name="x") 
y = tf.placeholder(tf.float32,[1],name="y") 
z = tf.constant(2.0) 
y = x * z 

print (x) 

with tf.Session() as s: 

print (s.run(y,feed_dict={x:[100]})) 
print (s.run(y,{x:[200]})) 
print (s.run(y,{x: np.random.rand(l, 10)})) 
print (s.run(tf.pow(x, 2),{x:[300]})) 


Tensor("x:0", dtype=float32) 

[ 200 .] 

[400.] 

[[1.5783005 0.30819204 0.260686461.8491662 1.0529723 1.7923148 
0.9828862 1.5377688 0.06250755 1.2727137]] 

[90000.] 


QuiTSfiGijrTai rglu^peb QrBLlQsurrijdSleb uu5ltn&)u5hsiT Qun§] ^eifldanjut^ib Lorr^lfl^ ppeyaeifleir 
<smq-<5urhi&<ss)<sfT rBLDLDrrsv 3 ^jd (Lpuj-iurr^j. ^gjGumssrjD ^i—ihjaisiflsbplaceholders- 

gou uiu<ssTU®£g<sorTLt). GjGhssrGsflso variables -go 606 S)pujQ]d(^LbQun§i <s 4 g 6 ir 6 ou^ 60 ^ 6 S)^ rsmb 
d^fo Q6U6mL^u5l(T^d(^L£i. t 3 i^rrsu&)} esrrows & columns ^q^d^Lb 

6 T6STU6S)3)d a^iDQsuessri^. ^(Tijd(&jib. £g)/ 5 <$/j iSl pd&6$)6ST placeholders- 60 ^j606S)60. Run-time s b 
L£>it§) 1 fild> j5!J6n&6h 6orj 60 ij < 3 i 6 $)ff, ^uuis^Qiu epebQeo rrqTj rfiiy^rTmstnidr&ib G)&6pi£§]6opfb(3j 
$]6S)6o Qufilgjib uuj6sru(^d}6srjD6sr. 


2.6 Tensor board 

Tensor board <6T6stu§i uebQeo^j Qi_6ih<9 : rTrr&{6fr ) d&>)6ti)i_u5)6V r£la(Lpib 3i6mdd(^iai6S)6n 
eoenrjui—LDrra eoenrjrsgi 2 _<$siyLD <s06i5) ^(gjib. dipd&essri— 6TQ]d > § > id3 : ,rTLbLq.6b xl, yl, c 

6T6S)i ld 3 Qi—esr&rrij&iQrijddlevo—ifileb r£l<3iLpr5§]6n6n&6mdd(i] iSl6ST60(r^Lcrr!Qj. 

f=xl.yl + squared(xl) + yl + c 
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= 5*6 + squared(5) + 6 + 5 


= 30 + 25 + 6 + 5 
= 66 

^j<suiD€S)fD si/<stf)/7L//_/-D/7<s <5U<ss) ij[£§ 1 tf.summary.FileWriter() 5763) i ld class uiusituQSilingi. 

£g)g/ 'tensorboardexample' <o 7 W/d Qumrflsv r5Lb(Lp63)i_m ^rnGurranpuj directory-<sv 6y (^folder-go 
a_0Si//Td50/i). £g)<$/D06YT ^Gsxssrg&j rglaLfxsqasifhssr stqijd&rhi&ssxsmLiLh (events & summaries) 
Gi&LSldtgj <ss)6ud(3)Lt). ^j^iQsu s.graph Qp<soii> Gusmrrui—LDrra sussiprs^i arrLli—UuQLb. 

https://sist.sithub.com/nithvadurai87/dbbel7d6036ea6188d9e581c3cbb2af6 


import tensorflow as tf 

xl = tf.get_variable("a", dtype=tf.int32, initializer=tf.constant([5])) 
yl = tf.get_variable("b", dtype=tf.int32, initializer=tf.constant([6])) 
c = tf.constant([5], name ="c") 
f = tf.multiply(xl, yl) + tf.pow(xl, 2) + yl + c 

with tf.Session() as s: 

summary_writer = tf.summary.FileWriter('tensorboard_example',s.graph) 
s.run(tf.global_variables_initializer()) 
print (s.run(f)) 


[ 66 ] 

^uQurrSfi sustnpui—pgidainsst rglpssxso <oT(L£>$ aSiL-Gi—mh. tens or board-go ^jiudsi 

iSl 63T 6U (TJj LD «L.l_«naTO)(U ^Glfld3i<SqLD. 6006port-GC (°£]£<SS)65T g£lUjd(3jLt). ^I^SST GhsUGlflltf® 

TensorBoard 1.13.1 at http://shrinivasan-Lenovo-Z50-70:6006 (Press CTRL+C to quit) gtgstu&h Qumssr^j 
<3i<5v>LDr5^rr<5V, tensorboard ^jiurddld Q^rrsmigp^ddlfo^i Gtssrirp ^ip^ld. url-db Q&ssTrrp 

umj^^rrsv 5U6S)tjui_Lb aitrsmuuf^ih. 

$ tensorboard --logdir=tensorboard_example \ 


dLpdasmL- 5U6S)tjui_Lb Tensor flow graph gtgstqj < 3 i<s$)Lpds;uu(i])Ln. nodes ldjdqjld edges 
GTSipi ld ^ptsm® 'SqLD&rhJ&Gsxsnu QutDPSl(tijd(3)Lb). add, mul, pow GumssTjDGSxsi] nodes gtgst^j 
^6S)Lpdauu(l))Lb. (gfi&n &6ssfl&) Q&iusvurr(tf<9i63)6fTd (& > f)ld(& ) Li). a,b,c gtgstu Quiurj ( o )< s / t < swl _ 
variables, constants ^dlLUGSisu edges 6763Tirg ^ssiLpdanjuf^id. Qisst&nsmpd (ajtSldt&jLD. 
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3 PyTorch 


Deep Neural Network-sir Q&iusburrQastnGrT ^pmusupp^ 2-psi]LD uzpQprrq^ susSlssnuurTssr 

&>lLi _ sshduGu PyTorch ^, 0 ld. £g)g/ (Lpanj/TsSlsir Q&iupssia ^rSlsifp^ipsir ^ujsijd (3j(L£ 

QpsoLC) 2_0Si//T«d5/jL/L.i_ ssiuprrssi sst uussiuiurrad Glarrsmu 6£0 library ^(pid. Torch 

srsmju($)Lb ^jiurB^u suiJilds.ppspids.rrsxT QprrtpuiSlsir ^upuussiuiNlsb ^(psurrssrGp pytorch 

40“' 


3.1 Tensors 

rfig^rjsv QpiLQsunpssiaiu Qurrtpppsussip ppsi^aierr ^ssissrppjid Qusir&mj sts$tuu®iL 
Glanshasossflsir suLfGiupnsir Q&iusbuQdlsirpssT. ^jrsp smuurrriddlsviLh torch.TensorQ srssiiih 
class Qpsoi£>rr& ptjGH&snsn Qi^sir&rTrjrT&i S-qrjSurTd&somh. dipdaismu ^prrpsmp^sv xl srspnh 
Quiurj Q&rrsssTL- &rrsS) Qusirarrij S-qfjSimd&uuuQsYTSYTp]. iSl sir ssrij [1,2,3] srspnh Id array- 

ssisud (o)<®/TS 337 i_ x2 Qissr&rrrj ^(psurrds.uuu^shsfTp]. £g)g/ 3 2 _piuLj&s$)SfTd Q <s rrsm®srrsnp] 

srsirupssmsb, ^psir size ld^Iul/ 3 srssr Qsueifluu(^)sussipd sirrsmsomi. Gldsvild sysiiQsurrqrj 
S-pjuiSlsir ud&ddlsviil). L/shs/rlstmu £g)/_l(5j) ^ssissrpssipiLjLh Float32 susn&uSlsv ^(psurrdipii). 
< 3 ]si]simpj £g )sbsorTLDsb int64 sussi&uQsb a_06i//T<s<s sSi^idiSlsisrrrsb, Qufhu T-d(p uSilsorra dlrShu 
t -go £g)/_l(/J) torch.tensor() ^sbsop] torch.as_tensor() srsirp ^rjsmiqsb <pprrsnpi spsirssipu 
uiusiruQpS] 2 _(ri)GijrTdasorTil>. ^sirsiSlijsmssu—U-iLb ssisuppi 2 _(psurrda,uuNu x3, x4 
Qissr&rrijaisfr spGrj iurr^lfl spGrr sussiauSlsb S-QTjsurrd&uuuQsrTsrTsvipd Grrsmsipli. 

STSsrGsiiprTsiT g^sijsfilrjsm® Q ussr# rrssi piy ld <s 4 psir ld^Iulj ([2, 4, 6]) 

QsusrrluuQ)ppuuiTQ)shsnp]. ^ipiGsux2-g>iLjLh x3-®oililL &*_l_i_ ^lusorrpi. 

spsiiQsurrqTj Glussr&rrqTjLCi dtype, device, layout stssiild 3 usmLj&smsmj G)upn&(pd(pLi. dtype 
srsirupi Qussr^rriflsir ppsii sussiaissnu QsusifluuQpp g-psyid. device srsirupi sp(p QLssr&rrrf 
CPU-sb ^jiurddld Qairrsssripfpddlpprr ^sbsvp] GPU-sb ^jiurddld QanTsssri^(pddlpprr 
srsirussip Qsusifluu®ppi lL. layout stssiild usmi_i s ^0 Q ussrarrpssiuiu r£\ssissrsij& ^ssilcussiu 
( memory layout) QsusifluuQppiLf). ppGurrssipd(p torch.strided STSSind ld^ussiuGuj ^jpi 
QsusrrluuQ)pp]Lb. G&rrpssissr (Lpiupdliurrai torch.sparse coo srsirupi uiusiruQppuuuQ]) 

SU(IT)§>lpp]. 

^Qppprra eyeQ, zeros(), ones() Gurrsirpssisu (LpssipGiu (jppQpmpssiLD ^sssfl, ^sssfl, 

1- gd ldlLQll Qairrsssru^sssfl ^dhu sup ssi p QarrQd&uuLlQsnsn suupsup^lsv 

2 — 0 su nd 0 &>l sirp sst. LSlsirssrrjrand() QpsoLh Id, 2d, 3d Gurrsirp smpsurd&siflsv random-^ai 
^ssiLorsp srsssr&ssisrrd Q&rrsssru ^jsssflaisfr 2 _(psurrdaujuLL(^sfTsnssr. 
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https://sist.sithub.com/nithvadurai87/79d938591a3ddf6d2234dc54c441082c 


import torch 

xl = torch.Tensor() 
print (xl) 

x2 = torch.Tensor([1,2,3]) 
print (x2) 
print (x2.size()) 
print (x2.dtype) 

x3 = torch.tensor([1,2,3]) 
x4 = torch.as_tensor([1,2,3]) 
print (x3) 
print (x4) 
print (x3+x4) 

#print (x2+x3) 

print (x4.dtype) 
print (x4.device) 
print (x4.layout) 

print (torch.eye(2)) 
print (torch.zeros(2,2)) 
print (torch.ones(2,2)) 
print (torch.rand(2)) 
print (torch.rand(2,3)) 
print (torch.rand(2,3,4)) 


r&j epidarrein Qeue^iddh: 

tensor([]) 


tens or ([I., 2., 3.]) 

torch.Size([3]) 

torch.float32 
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tens or ([1, 2, 3]) 
tensor([l, 2, 3]) 
tensor([2, 4, 6J) 

torch. int64 
cpu 

torch.strided 

tensor([[l., 0.], [0., 1.]]) 
tensor([[0., 0.], [00.]]) 
tensor([[l., l.J, [I., 1.]]) 
tensor([0.7487, 0.0652]) 

tens or ([[0.6830, 0.9479, 0.7002],[0.3283, 0.8602, 0.6711]]) 

tensor]]][0.8002, 0.9752, 0.4617, 0.5603],[0.1520, 0.6906, 0.9570, 0.7589],[0.0122, 0.8932, 0.9644, 
0.3375]], 

[[0.5123, 0.3771, 0.5494, 0.1664],[0.0154, 0.4539, 0.8266, 0.8343],[0.8994, 0.5009, 0.0348, 0.0757]]]) 
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3.2 Some useful commands 

topkQ srssrp aLfiss)srrd(psir k=l srssrprrsb (Lppsvasupi Qufiliu LD^Iusmuu-iLb, k=2 srssrprrsb (Lppsb ^psssr(£i 
Qufiliu LD&HuLiasinGfTiLiLb QsusfiluutjfippiLb. dipdaisssru 2-prrijsssrp5fisb X srsstuL 2d Qissrarrfi 
s_(ijGijrTdaijuLl($)erTerT§i. ^pptpsh dim=l srssyibQurrpi QarrQdauuL-L- 3 columns-g> 0,1,2 srssr 
ufilumsssruutjfipSfi, (Lppsb row-sb a_ srrsrr3 columns-sb QufiUu LDfifiuurrssr 120- ®» iqih, 2 supi row-sb a_ siren3 
columns -si) QufiUu LD^Iuurrssr 160-^U-iid Loppiid ^suprfilssr ufilixinsssrrh]aiss)smqLb Qsusfiluu(g)ppiSlppi. 
^ipiQsu dim=0 srspudQurrpi QarT®dauuL2L_2 rows-®o 0,1 srssr ufilLDrrsssruuQp&il, (Lppsb column- si) asrrsrr 
2 rows-sb QufiUu w^luumssr 160-^il/ld, 2 supi column-<sb a_ efretr2 rows-sb QufiUu Lc^luurrssT 120-%>uph, 
Qpssrprrsupi column-sb s_ srrsrr2 rows-sb QufiUu Lc^luurrssr 90 -^ilild urppjih ^suprfilssr ufilLcrrsssrrRiaiss)smLiLb 
Qsusifluu(p)ppiSlppi. 

Qussrarrfilsb £§) 0 / 5 ji/ 90 srsmestem lol(J)ld ifilfilppi srQda torch.Tensor().item() srssrupi uiussruQ&lppi. 

x.mul(2) srssrupi Qarr^tdauuiiM—STSssrsstsssr Qissrarrfilsb ssrrsrr ^ss)ssrppi LCiSluqansni _ spud Qu(pdSl 

QsusifluuQppiid. ^ipiQsu srpp 90 operation-sir udapGfisyiih underscore -®o GafidtajidGurrpi (mul_Q), ^§1 
QsusifluuQppiib qfiiUu LD^Iuqastmsb sppQassrGsu a_snw Lc^luqaiss)srT ^uwrrppih QaiLULfLb.. £g)/570 prut 
under score-®ou utussru/^p^ULisirsrrppasb gjfissflGSLDsb x-sir ld^Iul/ 2-^sb Qu(pdSld Slss)upp qfilUu 
LD^IuqasmsrrQuj Quprfil(trjd(3jLi>. reshape QaiLULiidQuapiLD ^LbuL^luqaiQsn Lop] sir upsu id 
Qa=iuiuuu($)suss)pd arrsssrsomd. 

90 Qissrarrssiij array-^a inrrpp ^pp Qissrarrfilssr Quiufi uda>p§ilsb qsrrsfil ssisuppi numpy() srsirupi 
uiussruQ&lppi. ^pjCSsu «^ 0 fl rray-ssisu Quest amjrrs, umppisuppua, torch fromnumpyf) srssr uppish 
Lnrrpp Qsusssnspiu array()-ss)end QaurQfidai (Ssusssr/^id. 

90 Qussrarrsarj mpisuq.suidfreshape) Qaiiieuppip view/) aulusosrr uiussruQfitdippi. ^prrsup] 
Qaur(ifidaiuuL/u2d Qussrarrsorj 1 d-^a mrrppisijpp(p, ^tpp Qissrarrfilssr udap^lsb qmoirl ssisuppi 
view(l,6) srssrd Qarr($idasvmd. ^prrsupi ^rrsssr(fi rows-sb ssrrsrr Qpssrp] Qpssrpi columns ^ssissrppiid spGrj 
row-sb suppisSK^nd. ^sbstnsoQiussflsb view(l,-l) srssrd Qarr($idasomd. -1 srsstiidQurrpi. pmd (ajrSluiShd® 
^ippssissr ^pjuqasrr srssrp] a^pp (3pss)suu5lsbss)so. ^ppssissriL/Ld iDpjsuupsuid Qa=tuiuuuL2($)G>5}($)id. 

https://sist.sithub.com/nithvadurai87/8109fea71fd03258a494d4b8ee727a9a 


import torch 

x = torch.Tensor([[110,120,90],[160,20,60]]) 

print (x.topk(k = 1, dim = 1)) 

print (x.topk(k = 2, dim = 1)) 

print (x.topk(k = 1, dim = 0)) 

print (x.topk(k = 2, dim = 0)) 

print (torch.Tensor([110]).item()) 
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print (x.mul_(2)) 

y = x.numpy() 
z = torch.from_numpy(y) 
print (type(y)) 
print (type(z)) 

a = x.view(l,-1) 

print(a) 

print(a.size()) 


r&ij<spidamzsT Qeu&fUtfCRl: 


torch, returntypes. topk(values=tensor([[ 120.], [160.]]), indices=tensor([[ 1],[0]])) 

torch.return_types.topk(values=tensor([[120., 110.],[160., 60.]]),indices=tensor([[l, 0],[0, 2]])) 

torch.return_types.topk(values=tensor([[160., 120., 90.]]),indices=tensor([[l, 0, 0]])) 

torch.return_types.topk(values=tensor([[160., 120., 90.],[110., 20., 60.]]),indices=tensor([[l, 0, 0],[0, 1, 

1]])) 


110.0 

tensor([[220., 240., 120.],[40., 320., 180.]]) 


<class 'numpy.ndarray'> 

<class 'torch.Tensor'> 

tensor([[220., 240., 120., 40., 320., 180.]]) 

torch.Size([l, 6]) 
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3.3 GPU 

CPU srsiruaji s$(nj asssflssfluSlssr Qpstnstr srssrprrsb, ^gy/jdg QpstnsmiSlsir Q^iusb^lpssissr usvLDi—rii(Sj <gi&Haifld(3jLb 
susmauSIsb ppGurrgi s-QpsurrdaLd ^nssu_!T,§>i(rrjuu(j>&, GPU ^qld. £§)ji/ Graphics Processing 
Unit sTssruuQid. Graphics, 3d games Qurrssrpsijpifilsb djsnrr s^Qpiiiassuns^ Q&tusbasmsfT ^S/iasnsy 
Q&iusupp^ib, Q&iup6S)a ^rflsypS/lpsir atfsmpasfilsb usbQsupj aipssr asssflp Q&tusburTQassiGtTgnflpiima 
Gl&iugl (Lpipdasyid ^l[Bp GPU Qu/flgiid 3-pei]£lp§i. ^ipssr gpflp ^ppsbprrssr ^lagiisiriLm ffipuGu! 

dipdasssn— s_&>rrrjsmd,§/sb random-^s, CopijrjjGpQdcjn 'it iC.t_ ^uSltjid rows & columns-god QarTessn_ a 
uoppjid b CTgn/LD gjfiijessr® 2d ^sssflasrr S-QpsurTdauuil^shsrTssr. idlssrssrij ^ssisu ^jijsssrffud matmul() yosvrd 
s^ssrQprrQi—rrssrpj Qu(pdsiiju(f)Slssrpssr. $}rsp Q&iusb psmi—Qupisupp^ (Lpssrssrrj 2 -srrsiT Qprgipid UIsstsstij 
2 -GfTGfT QpUQpib Qpes)p(Suj start uoppnd end srspiid £§)0 variables -<si> G>&idldauu($l£)ssrpssr. i3ssrssrtj ^jsnsu 
g£irjessr($)d(GjLC>rTesT sdlp^lujrrspssrpd srsssr(fn3ipuupsir fnpsvid $lpp Q&iusb CPU-sb pssu—Qupisupparrssr 
Gprjid asssrddli—UuQSilprpi. 

i3ssrssrrj piogi asssflssfluSIsb GPU ^(pddipprr srssrussip Q&rrSiida cuda.is_available() CTgn/Lo siLLaai 
uiussruQSlpgi. ^1^1 True srssr QsustfluuQp&ilssrrTsb, puss# a, b or spud Qissrsrrtj a&svsrr GPU-sb 
Q&iusbuQsupptGj sjpp susmauSIsb uorrpp ,cuda() srspud function s_psijSlprpi. ^ipssr Qpsvid ^smsu GPU-sb 
tsjppuuUL- l3gst iBsmr(fiid ^sdsdlrTsssr® ^sssflasiflssr Qu(ijda6£idarT6ST Qprrid asssrdSili—UuQSilpgi. CPU - 
sb ^ippauTssr Gprjid 22 sdlpmpash srssrprrsb, GPU-sb (s^ssrspud spflpuma 0.0002 sdlpmpuSlsb ggjdQ&uJsb 
pssu—Quppi Qpipp^lQpuussipd arrsssrsomd. 

2 -ihiasnsji asssflssfluSIsb GPU gjfisbstnsoQujssflsb, google colab or spud ^jssisssrussiuu 
(https://colab.research.soosle.com/notebooks/welcome.ipynb) uiussruQpGil dLpdasssn_ (S&rrpssissrsmud 
Q&iugi urrrfdasDmb. 

https: //sist. sithub.com/nithvadurai87/8eebce687ba439cd9b9691383c780dbe 


import time 
import torch 

a = torch.rand(10000,10000) 
b = torch.rand(10000,10000) 
start = time.time() 
a.matmul(b) 
end = time.time() 

print("{} seconds".format(end - start)) 

print (torch.cuda.is_available()) 

a = a.cuda() 
b = b.cuda() 
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start = time.time() 

a.matmul(b) 

end = time.time() 

print("{} seconds".format(end - start)) 


r&ijspidamssT Qeu&fjufCK): 

22.068870782852173 seconds 

True 

0.00020956993103027344 seconds 
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<class 'numpy.ndarray'> 


<class 'torch. Tens or'> 

tensor([[220., 240., 120., 40., 320., 180.]]) 

torch.Size([l, 6]) 
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4 Single Input Neuron 


^juu^^JuSlsb 2_GfTGf?L^($) ^QdSleb j^ls^urresxssunih, ClsueifhdLbf^ ^QdSilsb s^0 
rglu^rjmssxssTiLiLt) tssxsupg] aessfluiSlissxssr rflaLppgjGug] €tuul^ sTssr/ry unrjd&<somb. ^gesxssr it, mb 
tensorflow uiu<ssTU®£§il G)&iu§] urrrjdau QurrSlGprub. 



https://sist.sithub.eom/nithvadurai87/4ad051ef5c3cc2a3da6c043cd99d0flb 


import tensorflow as tf 

X = tf.constant(0.5) 

Y = tf.constant(0.0) 

W = tf.Variable(1.0) 

predict_Y = tf.multiply(X,W) 

cost = tf.pow(Y - predict_Y,2) 

min_cost = tf.train.GradientDescentOptimizer(0.025).minimize(cost) 

for i in [X,W,Y,predict_Y,cost]: 
tf.summary.scalar(i.op.name, i) 

summaries = tf.summary.merge_all() 

with tf.Session() as s: 

summary_writer = tf.summary.FileWriter('single_input_neuron',s.graph) 
s.run(tf.global_variables_initializer()) 
for i in range(100): 

summary_writer.add_summary(s.run(summaries), i) 
s.run(min_cost) 
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^eheififif) ^(fiddleb s^ehm rglu^rjmssr 2_<sh<si?Ll($) rod) l3 tj 657 weight 6765)177 ^snsi^^js^lesxssr 
$IGS)<sm£&j 3)6$T&ii <9istssfluiSlsts)stsr r£}aLp£aj]LC> eresT/gi ^r^QsurTih. $]rzJ(3j 0.5 6765>i to 9-<sh<si?Ll® 
LD^IuiShss)ssr <oT®£g]d Qamsm®, ^i365tii—6571.0 6765)177 weight 77(51 tj lS) 65)657 ^65)6mf§i 
rfl&Lpg^U-iGrrisrT &<stisfluLipredicts 6765H77 Guujfleb G3i£ld3ijuLL(gl6hm§]. 2_6S5765)70iur7657 0.0 

6765)170 Q<51JGlflltfLL® LCi^lULly 6765)177 QuUJlflsC S~<ob <SfT. ^6116^1g6557(ft 3 (3) 70 2_6 YTSTT 

GeunpjurrGi— cost eresT/gi ^65)7pd3Liu(fdljo3 > i. ^3,65)6573 asmddh— u65)Lpiu (rpes)jo uj rrssr square 
method -go £g)/5V0 U7U657u(fi^^H7f6hGmr77b. cost ld^Ilj if65)657 (3)65)70Tiu^703n657gradient descent 
<5TS5)i ld (5(5 &jleudi65) (5 G1—6573rrfit eb 6yj065)jo sufluSlsc G3UJebu(fi^^l eSU—eoruh. learningrate 
6765)177 ld^IulSI 65)657 (56573#parameter-^3 6T@£§]dG)arT655r® 100 3jd^J36ifl6b cost-god (3)65)7033 
(LpjDU@d)fD&i]. 

^ (p .$<$.$/t« eu7065)70 Guam eon to Gi—6573T7[t ^l65>puSl60 G6U6i?hju(f)^§]6u^70377657 rflpeb 

Q&rr(i)ld9iUULL($)6h6fT3)l. 677bG(575(5 77^177 7^365) mQlU 6060 7T77 &yl65)[7U e i}6V Gl6)J6lflUU($1*5*5 
6)fl (71) 70 7J Si GfOnG 70 T7, ^ 6U if) 65)JoGlUebeOTT70 6£(l]j list-d(&j677 ^65)70^3# for loop-QpeOTO 
Q^77i—7jdd)iur73iscalar_summary() 6765)00 function-d(3jeh Q&evid&iidjGfn 7770 . (f£](5657arguments-^3 
list-6b s^ehen eyehQeu77(H) Gi—657377ifl657 Guiuq^to, <3i&>657 dip ^65 )lh 75 ^]mm LodiluLjaiQrTjLO 
Q&nifid&iLjuif(i)i6hm657. <3i^rT6>J3# eyehGeunqp Gi—65737rfl6S7 d(7p77 <3]65>77r53#6hm 
LD^Iui^3i6ifl657 3(77)33 Q75flf(Lp65)foGuj (summary protocol buffer) ££l&65T G6U6iflufi—T73 
^65)77^1(03#. £g)/5;$ 3(71)33 5 G75j5)(7p65)J0361?l65T <31 ipijU65)l—ll5)6b 3(7^33LO GniTIUJljuifl— 
^(65)6573,3# Qt 657377!J365)miLjLD 6^(7^7hlSl65)6557U7J(5j6(3, Summary.merge_all() 7JUJ657U(j))dlj03#. 

iSl65765777 6£(77) session-go 2_0S1//T«® S£0/57® 65)655733 7J U ill— ^65)6573,3# GI—6573T7[T361?l657 3(17)33 
L£)^lLJLj3i65)miLjL£) 6U 65) !J U !—77 773 6U65)7J753# 37 Tl 2I— tf.SUmmary.FileWriter() UUJ657Ulf(fmm^]. (g£]3# 
£tnGuT765)guJ directory-eb ’single input neuron' 67657/o Quiuflev s^0 folder-go 2_06iy/T<S0/i>. 
££l&tD(3)6h grresr <s>I65)65t£§j G i—6573 it^ eifleh summary G&ttuli eoupefleb 

asynchronous (ipesijoufeb G&Tjdanjufj/LD. iflehesr/j for loop specif 100 3jd^j365>m 2_06iv/t<s5®, 

6j>671G67117(17) 3jr)f)l60l77 0.025 6765)170 370(060 6)11^13,3,^1657 ^IgLlU65)1—uf 60 ^65)7777,3, LjG^UJ 
parameter 77^lui#3ipfpd(^ summaries-go 2_(7T)6iJ rrd&il (ff65)6S57d£lj03#. Gi—65T3mj (gleeiTjuleb 

6Ll6S)rrUL _ 3,65)3,3 3T76S57U^jf)377657 uS(l6)jd G3TTULj363 (logfiles) ^65)657^^177 

to(L p65)jo u5) Geo Guj s^q^eu 77337 ) u(f) to. 

'3t®(5(5(57T3 dip336S57l— 3if l _ 65)m65)UJ ^jUjddt Gl—657377lj ^I65)gu5l60 G3657QJ U T7rj3361J 70. 


TensorBoard 1.13.1 at http://shrinivasan-Lenovo-Z50-70:6006 (Press CTRL+C to quit) 
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$5l<ss)[ju5)so Scalars lcjd^JLCi Graphs <oT<s$)iih LSlflsyash amsmuu®LD. Scalar <oT<s$)iih 

LSlftaSleb list-d(3)<sh rsml) Q&rrQp&i] Q<sij<sifluu®£gd Q&rT<sb<sSUu s^djQsunq^ 
m§}uLig;((srijd(&jLDiT6$T sustnrjui—LD iShsirsuQ^su^QurTsb 3>rr<smuu®ii). <oT®£§]d<9HTLli_rT<3; 
predict_y ^rnhjdULjenen ld^Iul/ 100 &Lpjndl<3;Gifl<sb, s^sijQsurTQ^ &Lpjndlifil<spjLb 0.0 OTgo/fi) 

^stskrsts)LDUJrrssTy- ld§)]L iLjd(<9j QrBQTjdagdilsb <srsijsijrrgu (&,£$)ro[Eg] Q«/tsw(3/_ siiQ^dlfD^] <oT<s$tu§i 
suanprBgi arTLh—UULL®<sh<sn&ii. y($)£pprTai Graphs <oT<s$)iih u^tsluSlsb 
Qi^G&T&rrijaqirijddUstni—ifilsb rs<ss)i—(olu/D/D 3nsssrdd(^\3i^d3imsst GU&npui—LD Guemprs^j 
<s ml i_ u u lL ($) Ghtsrr^]. 
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ct)-> g ft 


© localhost:6006/#scalars 


9 0 vo 


TensorBoard scalars graphs 


I I Show data download links 
Q Ignore outliers in chart scaling 

Tooltip sorting method: default 

Smoothing 


4.1 

Horizontal Axis 


WALL 


Write a regex to filter runs 

DO . 


^ Filter tags (regular expressions supported) 

Const_1_1 

Const_2 

Mul_1 


! □ 




80 100 
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5 Neural Networks 


Q&6STID CT(J)< 5 ^«<S/TL/Jj_<SU ^GfTGfTli® ^ (^ddlSO 2 _<sh<SfT S >£0 fffu^rTfT&SVSVTlLILb, QSU Gift u9lL® 
v 2_mm ^0 rfliLjjfjrTGSxssTiLiLC) ^esiessrg&j &<sssfluLj ersijsijrTrrg rBi—ddlfDg] <oTesrru 
umj^G^mb. ^juQurr^] erreifii.($) ^Qddleb use rfli^rjrTesr&esxsn ^ssild^^j ^<sujD<ss)fD 

QsuGiflitfLl® <3]®ddl<sv 2_<sh<nT s £0 ril^mrestii—esT Oftesiessrd&i aessfludilesiesT ereueun^j 
rffl <s Lp£5&n<5u&n (oTgstqj urTrjd&eomi). (Lp^eSleb ^j^tssr ^^leurrrj^rhJaiesxstT &rrg,nrj<5m ss)u^rrssT rflrrsv 
Q&rrem® erqgSilu Ljrflrsg] Qs,rr<5rrGeiirrii). iSlssrssrrj ^< 5/00 ^esiessnurresr Qissr&mj rflrrsmsv 
<OT<51J<51Jrrgj uuj<s&ru(l))d>&f]6u&ii GTesrrgj urrrjdaeorTLC). 


5.1 Python code 

rig^pev QrBLlQsumjdlsv 2 _<sh<sn uecQsu^j rfHu^rjrresT&Gh umlplrflg ppeiiaissrpi—esr ^eufDfrpdGa 
s^fliu u<svG)surry ^erreijQijdaesiGrT ^es)essr£gi ^ssxsmg&j &estfluLj&es)Gn rglaLppajjdlfDgi. 
(LppsSlsb <s>l<srT(5ii(Ti)d&a<sifl esr (parameters -weights, bias) LD^lUL^anssxsn 3JLfluJLt> eresr esxsupsji 
ansssfluL-ignssierr ril&Lpptglu urrijd^id. ^jrsrilesisvuSlsb (3j<ss)fD[5& ^errey aessfluL-iaGerr 
&rjUurTat5ijLh, rilesi/Diu &estfluLj&Gh geurnrT&enih ^esiuurjih ULi&pjrsleb ^<sn<sn(njd&i<sfil<ssT 
i£>§)uLj&<ss)<sn um/Djifl LimfDrfil &rfhurrad ajesrildsi (LpiuebdlfD&n. a&ni—dhurra e^penei^d(^ 

&><S$$flULI<3>GYT 'SHG$)(5$T'^3tfLL tflflUJITSi UL^&d,&>l<sb, <$6370/ ajDJDSSKSO rifrj]d,S)ld 

Q&msrrdlrDSji. £g)<$<$63><suj <s63>/_£) riles) evuSleb rsmb uujesTu^^^lu^ehen ^enei^q^daieilesr 
i£>§)Iu<ss)uGuj <5T§i)[jarT<5o£§i)<5b eupuGurrdlesrjD ^pei^ajes)en &esrituu$in(3) rprrih uiuesruQgdild 
Q^rrerreneomb. 

63)63700/ umSilrflg ppeiiaeiflesr 2_<sh<si?(i))a<ss)<5n stsxsu^gjd Qs,rremQ) esr GhsuisrrlitfQaissKsrT 
&rfliurr&d &<55tfluu£iD(& ) , arLflujdt&tKsb ^rndiSldtgj, &es)i—§l rilesieoesnu ^esu^iLjih euesirj 
<3i<5fT<sii(rF,d3i<5i?l<5ST ld^Iugsiu LDrTjDffil LDirjujil aessfld(3jil> (LpesiroGiu "(LpesrGesTrrddl u upeypeb" / 
Forward propagation eresrrrp ^esiLpd&uuQSljDiji. ^erreijQTjd&GiflesT ui^luesiu epp^dlev 
GrrrjessflaeiflesT ^Lq-Uuesii—uSleb umjDfyjLC) (Lpes>/Dd(3j iSlesiGesmd&tlu urrei/^eb /Backpropagation 
eresrgg Quiurr. ^eiieurTrorrs, (yiesrGesrrrddlu iSlesiGesmd&tlu urrei/^ev ^Lpeoib rflu^rjeb 
QrBLiQeurrijdarTesTSfi 06370/ <s/D/D63>6U rilaLp^gidlrog]. 
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dipd&GmL- <oT®£g]damlLii-<5b 3-<sh<si?Li(b)£ ^rrsif s ^0 2d array-ssxsud GlamsmQshsn&j. 
^IGUfonSlsb 4 rows ldjd^JLD sysuQsurrq^ row-s/f solid 2 columns ^ehsrrssr. <S43 > rrsug > i 2 features-^eb 
G&snd&uuGfnh 4 Lorr^lfl^ <$/ 76 i/<sffln 6 yr<s QarrsmQsrrsrTgi. £g)/ 5<5 2features-id a_ eheifLl( J) 
<3l($)ddl<sb ss) ld iLj ld 2 r£l ty, rjrrssT3surra ^smDiLjih. ^jpfDamssr Glsusiflitf® 0 LDro^jih 

2_<sh<sfT§i. <st<ssiQ<su ^3)1 logistic regression-d&nssr ST®£§]d&mQ® srssru^rrsb, ^pfoarresT 
G)gu<si?Iii9lL ® layer -60 sigmoid activation fn uiussruQi^^uuLdQiSTTsngj. 



https://sist.sithub.com/nithvadurai87/75e91a68e07d2e375aaf3ba4c87ef49a 


from sklearn import datasets 
import numpy as np 

X_data = np.array([[0.4,0.3],[0.6,0.8],[0.7,0.5], [0.9,0.2]]) 
Y_data = np.array([[l],[1],[1], [0]]) 

X = X_data.T 
Y = Y_data.T 

W = np.zeros((X.shape[0], 1)) 
b = 0 

num_samples = float(X.shape[l]) 
for i in range(1000): 

Z = np.dot(W.T,X) + b 
pred_y = 1/(1 + np.exp(-Z)) 
if(i%100 == 0): 

print("cost after %d epoch:"%i) 
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print (-l/num_samples *np.sum(Y*np.log(pred_y) + (1-Y)*(np.log(l-pred_y)))) 
dW = (np.dot(X,(pred_y-Y).T))/num_samples 
db = np.sum(pred_y-Y)/num_samples 
VI = VI - (0.1 * dW) 
b = b - (0.1 * db) 

print (W,b) 


rfiijepid&rresT aSlendaiL: 


LDrrtslffig ppeyaerT (Sample data): 

dipdasmi— 2_^rrrr<smf&>l<5b &_tsrreifi_rr3i 4 LDir^lfl^ ^/rsiyasisyj/i) (Xdata), ^i^jn&.rrssr Ghsueiflufi^nsi 
4 urnGjliflp pijGijaqsrijLD (Ydata) uifilfD®d(&j ^sufldsiuuLdtfimm&ST. £g)<$<sir Qsuerrhifi^rresr^i 0 & 1 
&7W £§)0u l/<s rrsb, ££]§] logistic regression-d&mssTgi <oT<ssr rsm! QgtflrB&j Q^rrshstreomb. 
^erreififtfldsimssr ^psumssr^j 4 rows & 2 columns QantsmL- 2d array ^eherr^i. <S 4 &,rr<oiJg > i 4 
sample data-damssr2 features Q&rrffld&uuLdQmmgi]. 

Xdata = np. array([[0.4,0.3J, [0.6,0.8], [0.7,0.5'], [0.9,0.2]]) 

Ydata = np. array)[[!],[!],[1],[0]]) 


a-erreifLi(Sl (InputLayer): 

rflu^psb QrBL-QsiJrTijd-ggu Qurr(rij£g euss)puflsb (Lpgsb ^(fidSlsb a _erreifif (fits jBtJGy&^d&mssr 
features <S4<s$)LDr5$5l(ri)d(&jL£). features- ssr <aremsssfldes)3iuflsb <S4<s$)L£>r5& weights & bias ^essfl 

2 _ 06 i//t<s<sl/l/l_(J) 3iessrddQ)3i<sh iflai(L^Lt>. ioTgstQsu 2 _<sfT<X?if(fid > ppisyassxsrT transpose Q&iu&n 
features*sample records srssifQi ^smLDiLjLDrrgu LDrrfDffld Q&mshm Qsu<sm(fif (2,4). ^sijsurrQ/D 
QsueifhififQ)^ ^p<si^3iss)snd Q&msmL- ^essflufl ssr shape-if (4,) <oT<s$r ^(r^d^if. (ffon^upd 
transpose Q&iug] ( 1 , 4 )<oT<sst LDrr/Djfld Q&rrshsfT Qsusm(fid. ^jdjsurrGfo X, Y^(r^surrdsiuut^Slp^]. 

X = Xdata. T 

Y = Ydata. T 
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'g/errey 0<s<s eiffiesr 3fieuda$GS)eo LD^IULj^eh (Initializing weights & bias): 

^juQurr^] X.shape[0] svsirugji 2 features-go U-i ld, X.shape[l] GTGsrug] uu5)fD@)d(3) 
^eifldsiuuLLf^shsrT 4 umtsliflg jBtjGy&ssxsnu-iih (^fSldSljD^i. weights-go 0 gtgst initialize 
Q&iUGiJgfD(gj np.zeros() uiUGsru(f)SlfD§]. £g)/ 5<5 <zgV<5U0 LorrShfif gpeyaGiflsb 2 _<snw features-sst 
Gremessfld®s)&;d(&) ^ssxsmujn&i ^) 0 <s<a (Ssi j<sm(f\ih <oT<ssfu&,f)&,n&> X.shape[0] gtgst 
G l&rrQdaiuuLl® y®g®5)(/j <si/sssfl s^Q^surrdaiuuLl^ehsn^i. bias-d(&j gjGijda ld§)li urra 
Lj^gShuLD ^GifldauuQdlfD&i]. 

W = np.zeros((X.shape[0], 1)) 

b = 0 


earruptEiaefr (Epochs): 

X.shape[l] <oT<sstu§] uuS)iD@d(§ <s4<sifldauuil(j))<srT<srT 4 Lorr^flf ^rrsy&sinmd (gffldigLDrTpcsorTsb 
(g£]6S)g (ssxsugaj] num samples GTSVinf variable S-QTjGiirTdauuQdlfD&i]. £g)g/ cost s,<sm(])i3i^ds,ij 
uojsstu(^SI(d§i. <S4prr6ijgji ^p6i^3iGifl<sb <oTsi]Guerrey ^/rsi/«(sy5«0 ^usssfluLisush^suprr&i 

r£l&Lpr5§]<SYT<SfT§] <oTGSTUG$)^d &><smd&n_u UIUG$TU®£1 iD§I. (Lp^<sfi<SV Lj^DgShULD 6TGST 
<SUGS)pUJ^jd3iUUllt __ ^mGipjTjd&GSXSYT 6S)&G$$fluLI&G!S)m rglaLp^S) cost 

&<sm(i)iiSlu\-d§il(D3)i. £§)/5<5 cost ^ £g) 0 <£ 0 /i> ulL^^Igo (^pmsugi <s>lG)a 
susssfluLisush ^(suforroi rflaLprB^QTjrB&rTGo) Q&rrQfid&iuu iXl_ ^gwsii cj^dsissxsfT LDnrorrf if(sm(f)Lb 
asssflggi ^gjoamssr cost asm® iSh^ddlingi. GUGurrQfD for loop ^Lpsoih 1000 SrfBirp&.m 

G)&<sbd)6STfD6ST. epsijGhsijrTqjj ^jdqjlL 1 epoch / &arrugLC> <oT6STfD6S)Lpdauu(j))d}fD&i]. spsuGhsunqi) 
epoch-sol if (LpesrCSesw-ddlu urrof ^rr<sij&<sh ^Gtsxssrd.QViff.u^Lb a<sssfld(3)LC> (LpesipiLjid, 
aGssfld&uuLlL- ^/rsiy«(S)5<s«/TS3r ^Lpusnud a<smd§il(j))Lt> (LpesipiLiLD , £g)ipuL/ <3i§>l<3iLDrr<9i 
(afiqtjdt&jih uiX&f&yUsv iShssrQGsrrrdSlu upsi/^GV (jp&Dfouuui ^m<sii(i^d<9i<sjv)m LDrr/Dr^jLD 
rfl&LfxsijLC) Qgrri—[rd@)ujrTa rBssyi—Qu^jdlesTfoesr. 


QfidrGissrrTdSlu upsi/pso (ForwardPropagation): 


42 



6£0 flu^pmssr inputfeatures-sc ^sherr L£>$5luLia(<sriji—Gsr weights ldjdqjiL bias-go# G&rjggi ^<s$t#j 
3isssrd^Q\3i<ss)<sn^ Q^m_rhi(# ) il) stsst <gr/DQ<#stsrGsu urTp^G^rrid. ^jrhi^Lb np.dot() ^Lpsoih 

2-GrrGi?Ll®£ #>/ 76 i//_ ssT 3 tl£IujlL stssiill g/sud# LD&)lu®nuu Qujd/d weights ld/d^ld bias-go 

63)63373^5/ z -g?<® & t sssrd&>l(])i&>lfD#}]. £g)/5<$ Z 6 T6$TU3j] spqjj rflu^pnsir <956537«S)ll_ linear (LpesipuSlsb 
tSysis)L£>fE#> #S5$fluLi<#sh ^(3)LD. ^)^63>63T logistic regression-d(3j SJJDJD (Lpsm/Buflsv < 3 i®S)Lbd#>, Z 
ld§>! Li u rrssr sigmoid activation function-d(3jsrr Q#svid#>uuLffRi 0/1 st&st #sssfld#uu(f)d/D§]. 

£g)G>.$ GurrssTfQ] sgshQsurrq^ (LpssijBiLjih, umfo/DuuifL- ^snsi^q^daissxsn ^snsmpgi 
<S 633^77_77_/<S 63) 6)T f}<9iLp£§l LD (LpSS)pGlU forward propagation 6T6377ry <3ISTS)Lpd<#UU(f)§>lfD#}]. 

Z = np.dot(W.T,X) + b 

pred_y = 1/(1 + np.exp(-Z)) 


jg)Lf>u<oS>ud ssmddQgeb (Finding cost): 

SrLfhuLb stsst ^g)/6Y76i/0<s<s6)T (parameters = weights/bias) ^(r^d^if Gutt§], g)0 rflu^rrrrsir 
asssflddssrjD sfls^iuif psuforra ^iss)ldgij^/d<#ttsst sumuuL-i snf,#, ^isns^dt#) 2 _shsn§i stsstussi#,# 
aernddlLl® Qsusi?luu(d^§]Su^fD3iTTSST rflpsb lSIsst suqjj ldtt^j. £g)<3.$ Guttsst^j s^suQsunq^ 

#rff)f)]<SVILb <J)/6Y7'61/0<S<S6)T LLfTfDfDUUlf(f), #SSsflu L/SiSYT f) LJ U lf($) £g)/5<$ COSt 

asmddi—UuQdingi. QLorrf^if 1000 #rfdirp#srT stsstu^ttsv sgshQsurrq^ (Lpon/oiL/LD 
^jLbiD^ULi#sfr print Q^iuujuu/fsiiss)#,#, ^aSlpdai, #rfDfrp#sfflssT srsmsssfldsm# 100-ssr 

LDl _ /57<9>/T<5 ^j(T^d(^LCiGuTT§] LDlL@GlC) £g)<$63)63T print Q&IU& STSSTd&^fD if-gou 

uiussTudf^nLjehGefTmf. £g)<$63 tttso Qum^if 10 (Lpssip ldlLQGld cost Qsusiflu51i_uu(f)dp§i. 


if(i%olOO == 0): 


print/"cost after %d epoch: "%i) 

print (-1/num samples *np.sum(Y*npdog(pred_y) + (l-Y)*(npdog(l-pred_y)))) 


i3ssTGssmddu upsypso (Backward Propagation): 
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sjnjQ&issTGsu ^sherr ^ensi^q^daeiflsir ^ls^uussu—uSJgc ^jlcild^Iuli lSIsstsuq^lci sumuuurr® 
QpsvLC) asmddli—UuQdlfD&i]. ^jLDQpssiiDuSleb ^sns^Q^da^darrssr delta /derivative srss)iil> i£\ad 
dlnfliu ld§)1uli asmddli—UuQSilingi. derivative srsirprTsb Qurgj^ (Qpsog^ssflsirtgj Quigjd)sirro 
ld§)I liiLi) <3isvsv§] sijLflgGgrrsirfDsb STSSTigj QurrQjjeh Q&rrshmsomf). Gradient descent stssiild 
l / 0 ^) u 5 )<su ^jLDLD^UL] srsiisu rrgj asmddl i—U u ®d)fD3j] srsirussipd asmGL-mh. ^ff.rrsugj 
^sssrssiLDUjrTssi LD^lLiLjd^ih, asssfldauutli— LL^ULidi^Lilstsii^Guj ^musurrstsr spqij Gan® 
6U6S)ijuJuu®§iliD§l. ^id&mii6i)d GarriXipsir ld^IuGu derivative STsirpssiLpdauu®dlp§i. 
^jssi^d asmddl®6ii^fDarTssr snmuuurr® iSlssTsncj^LDrTgp. 

dW = (np.dot(X, (pred_y-Y). T))/num_samples 

db = np.sum(pred_y-Y)/num_samples 


<3i6fT6i](Tt)d3>ss)6fT Gimhu®£§igsb (updatingparameters): 

rBmd Qan®^§i6fTsn learning rate-^sc back propagation {LpsoLD rBirid &>smt_ffiliT,^ QuirpS) 
LCidiluLiassienu Quqrjdd), isjjDGlassrGsu s^eherr ^STTSi^Q^daeiflsir LuSiluLSlsSlqjjrB&ii aLflp&j Lj^lui 
tSyerrsiprijdassierT rsmb s^q^sarrdasvmJ). learning rate srsirugj srrsp ^g)/OTSiy«0 dlfShu^rra rsmi) 
*$\is\-6$nu st®£§] ss)6iida Gsnsm®Lh srsirussi^d (^rfld^LD (^jlip g 0.1 srssrd 
QarTsm®6hGsnmi>). 

W= W-(0.1 * dW) 
b = b- (0.1 * db) 


affliurrssr ^tsrrsnQ^dassisrrd assisr®iSluf.^sb(Finding right parameters): 

asni—dhurra assii—ld) ajonSlsb uiusiru®£guuili— ^srTsipnjdaeiflsir ld^Iuli 
Q6ii6ifluu®^uu®dlp§]. ^Guj ST&ylrrarrsD^&ylsv enpuGun^ih grjGyassierrd 
asssfluugjnt&ju uuisiru®^^ld Qanehsnscmh. 

weights = [[-3.44] [4.40]] 

bias = 1.66 
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[jjlijepld&rT6ST Qeu&fUdCHl: 

cost after 0 epoch: 0.6931471805599453 

cost after 100 epoch: 0.5020586179991661 

cost after 200 epoch: 0.4448439151612328 

cost after 300 epoch: 0.3979115275585397 

cost after 400 epoch: 0.3590456846967788 

cost after 500 epoch: 0.32654805177827606 

cost after 600 epoch: 0.2990946818904699 

cost after 700 epoch: 0.2756668906115973 

cost after 800 epoch: 0.25548229634006936 

cost after 900 epoch: 0.2379373586492477 

[[-3.43906374] [4.4016814]] 1.6577403314904984 

5.2 TensorFlow code 

G>LnfD&G$sri— < 31 G^ TensorFlow ^Lpscif ^(Lg^lLLjshm rflrr<sv iSlssTsuq^LcrrfQj. 

G>LDfD&<5mL_ <31 G^ QGUGiflu]if <s$)L_ Q su eifluSl($)if. £g)l|i) sc uiu s$Tu(ff§>l il/ <sbmfunctions- 
ssr Glpsiflsumssr sfilGrrd&rhJ&esxsn shallow neural network <oTss)ilci u(^^luSlsc s,rr<smscrrif. 

https://sist.sithub.com/nithyadurai87/fcl719d485dddfc4988c9e36975913c3 


import tensorflow as tf 

X_data = tf.constant([[0.4,0.3],[0.6,0.8],[0.7,0.5], 

[0.9,0.2]],name="input_value") 

Y_data = tf.constant([[1.0],[1-0],[1-0],[0-0]],name="output_value") 

X = tf.transpose(X_data) 

Y = tf.transpose(Y_data) 
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W = tf.Variable(initial_value=tf.zeros([l,X.shape[0]], 
dtype=tf.float32),name="weight") 

b = tf.Variable(initial_value=tf.zeros([1,1], dtype=tf.float32)) 

Z = tf.matmul(W,X) + b 

cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=Z, labels=Y)) 
GD = tf.train.GradientDescentOptimizer(0.1).minimize(cost) 

with tf.Session() as sess: 

sess.run(tf.global_variables_initializer()) 
for i in range(1000): 

c = sess.run([GD, cost])[l] 
if i % 100 == 0: 

print ("cost after %d epoch:"%i) 
print(c) 

print (sess.run(W),sess.run(b)) 
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6 Simple Neural Networks 


£g)g/6i/63>/7 rsmh umjd>&, ^ss)ssr^giLD L//f?/sg/ G&ttgttgtt SrSOULDrrsi QsusmQ)ih 

<oT6STUgfD&rra,, 1 rglu^pmssr, 2 rflu^rjmsir GTGsrg y dlfShu (orsmessfldemaiuSlsc GT®£gjd Q&iu&h 
urr/j^G^mb. ^juGurraj] ^GmGtnLDiurTaGGu 30 features-sc <3i<ss)i£>iLjih 426 uuSIjd^ rrsij&ssvsrT 
<oT®£§j <sp(ij rflu^rrsi) Q/5l_Q<siv/777<9563><s 2 _ 0 <av nddl u urrijd&iu GurrSlGgirrih. $l$<sb Qguqjld 
^gttgiT® ldjd^Jld Qsueiflid lL® d&mssr layer-go ldlLQGld QasTTGSsrigq^d^ih. ^)ss>/_u5)sii; GTrB&Gpfjj 
hidden layer-id &msmuui—rT§]. 


dy)ds,smL_ GTQjB&jid&mf igsc sklearn-d(3j<sn ^errerrdatasets GTGsru^ffx^GYT syq^suq^d^ LDrrrru&iu 
LIJDQjGlBrTlU ^IQTjd&rT (ffjGVGlDGOUJrT GTG$TLJGS)&) (LfiLgSlI Q&lUGljpfD&rTSST LDrr^lfl^ ^rjSH&sfilGST 

Q$n( 3 )ULi&<sh 2_<shm6$T. ^jesxsu 426 rows LDfogjid 30 features-god Q&rrsmi—g]. ^gidgii 
train test split ^Lpeoid iSUfds,uuif(f) normalize Q&iuuJUuQdUssTfDGST. ^sa/n/flsv <sp<r^ urr§>l 6 ti)ujd 
QarrQfpgl 4000 arfDjrpans^m^(j^Giirrddl, ajnr)6£jda;rT<s$T si5)®^sa>^ 0.75 stsjt &s>Giif§] 
QgifQisurrijd^u uuSlrdd) ^Giflp&u erflmmssr ^sttgii p^dai&DsrTd asGssr(f)iShgdS]Gg)mh. iShsirssrij 
ascrnffiShg^ ^msydansinm Gsxsug&n uu 5 lg)ld)d(aj ^Giflpp &>p<si^ai<ss)<smLjiL(X_train), srfld&rr&, 

£P<sn&<ss)<stnLiLb(X_test) asssfld&d Q&rTsbsgidlGjDrTLC). uuSljD^d(^ <s4Gi?l£p ^/Tsiyasyfisw 
§I<sb<s6hu^^sirss)i£i 98% gtgstgiild, uuSlfd^ld^d Q&sypgrrg lS^I^ ^/rsiyasaxar GT^lfairTGO^ 

<5/761/<S6U/7<S<S d50^) ^<$63)63735 asssflpgl GllQTjLD gJsbsSl ILIggGSTGS) LD 93% <ST S37 <31/LD 
G) Gil Gift UU(^)GUGS)^d 3> TTGtSST 611 ITLD. 
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Input Layer 


426 samples 
via 

30 features 



https://sist-sithub.com/nithvadurai87/6c39697e3134a7ba5bd5cbe8790f95db 


from sklearn import datasets 
import numpy as np 

from sklearn.datasets import load_breast_cancer 
from sklearn.model_selection import train_test_split 

def normalize(data): 

col_max = np.max(data, axis = 0) 
col_min = np.min(data, axis = 0) 

return np.divide(data - col_min, col_max - col_min) 
def model(X,Y): 

num_samples = float(X.shape[l]) 

W = np.zeros((X.shape[0], 1)) 
b = 0 

for i in range(4000): 

Z = np.dot(W.T,X) + b 
A = 1/(1 + np.exp(-Z)) 
if(i%100 == 0): 

print("cost after %d epoch:"%i) 

print (-l/num_samples *np.sum(Y*np.log(A) + (1-Y)*(np.log(l-A)))) 
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dW = (np.dot(X,(A-Y).T))/num_samples 
db = np.sum(A-Y)/num_samples 
VI = VI - (0.75 * dW) 
b = b - (0.75 * db) 
return VI, b 

def predict(X,W,b): 

Z = np.dot(W.T,X) + b 

i = [] 

for y in 1/(1 + np.exp(-Z[0])): 
if y > 0.5 : 

i.append(l) 
else: 

i.append(0) 

return (np.array(i).reshape(1,len(Z[0]))) 

(X_cancer, y_cancer) = load_breast_cancer(return_X_y = True) 

X_train, X_test, y_train, y_test = train_test_split(X_cancer, y_cancer, 
random_state = 25) 

X_train = normalize(X_train).T 
y_train = y_train.T 

X_test = normalize(X_test).T 
y_test = y_test.T 

Weights, bias = model(X_train, y_train) 

y_predictions = predict(X_train,Weights, bias) 

accuracy = 100 - np.mean(np.abs(y_predictions - y_train)) * 100 

print ("Accuracy for training data: {} .format(accuracy)) 

y_predictions = predict(X_test,Weights,bias) 

accuracy = 100 - np.mean(np.abs(y_predictions - y_test)) * 100 

print ("Accuracy for test data: {} %".format(accuracy)) 


rfirrevidarTGisr Qeue/DidCK): 

cost after 0 epoch: 
0.6931471805599453 
cost after 100 epoch: 
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0.24382767353051085 


cost after 200 epoch: 
0.18414919195134818 


cost after 3800 epoch: 

0.06044063465393139 
cost after 3900 epoch: 

0.05993526502299061 

Accuracy for training data: 98.59154929577464 % 
Accuracy for test data: 93.00699300699301 % 
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7 Shallow Neural Networks 


Shallow <oT6$TjnrT<5b ^LpLDfDfD gtgstqj QufTQ^sh. deep (oTGSTjnrreb ^Lp ld rnssr gtgst^j Qurrq^eh. gtgstGgu 
D eep fuprjGO QrBifQGurrrrGSi&u ujDffld 3>rQU& > [Q(& ) (jpGSTGsrf shallow fuppcb 

G)r5LiQ<5iJrTjj(SS)au ujdpSI^ QpfirB&j Gl^mshQsurTLCi. $l&fD(3j (Lpesresrp rsml uiuGsru(£)£$uj 
LDrrrru&u njDrgjGrBrnudarTGST g-prrijGssrpGSigGuj £g)/570/i> uujGsruff^fd Q^msbQsijnih. 

^ssrmsv ^psir 2-GrrGiT® LDjnrpjLD GlGUGiflufif® layer-ddlsni—ufsb hidden layer syssripi 
amsmuu®L£i. ^tglcsb tbttld <snss)pujQ]d(^Lh <srsmsmfldss)<9iufl<sv ^gshdts^ nodes-go u 
G)ufDrr5l([ijd(ajLt). ^j^gst GuifGuj Q&uJGbu®LbGurT&ii ^ gttg£\go mssrfeatures 0 gs)jd[b& 

GTsmsssfld <o5)«t/5)<su LDfr/opu uif(f) s,GssfluLja,Gh f^Qpif. 

dipd&GSSTL- frrsSlGV Lnrrffld, pjjGyaiGrT traintestsplit ^Lpeoii iSlfld&uuiflfi normalize 
Q&lUUJUU®GlJ£fD(3) (JpGSTGSTlj, GUGSlIJU l_LD GyGSTgp 61/63>/7770/ UTTljdaU U®£lfD§l. ^Sd^lGO (Lfi^GC 2 
features-go GT(f)f§]d Q&rrGm(fi (X_cancer[:,0], X_cancer[:,l]) ^GUforflGV 2 _GfTGfT pijGijaGrr 0 ^gogo^i 
1 gtgvilI) GUGSi&uumfLpGb (c=y_cancer) gtgijgijgs)&u5)gst dip ^gs)LC>§Iigst(dgst gtgstu§] (jpGSipGiu 

«0/i>L/OT6yft«OT/T<S6iyLD, L£>(6T06)T L/SlTSyfidSOT/TdSSiyLD GUGSipf^] &mfl_UU(f)&>)rDg]. ^]GS)GU 

Gusmrrui_f&>lGb GrBfGarr® (LpaDpufsb iSliflds; ^lUGorrp gpGyaGrmgid 3iGCiffqpuuGS)^d 
3HTGSSTGVmt). gtgstGgu GTGifl uj logistic regression-go u uuJGsru(f^§iGU§] 2_<$6i//70/. shallow 

neural network QaiTTGSsrQ) ^^gsigst tbttlci iSlfld&iu GurrSlG/Dmli. GldgvilI) ^gujdgs>jd tensor flow 
um g$t u(j))d>§)ld Q&luuju Gutt^iGjottlci. Q&gsttd GTtfifgtid&rrifipGb rsmi) ^gs)gst^gs)^iljiL 
GarnlurTQaGrTrTa fierce GT(Lpf Gl&iuGprrLb. ^uGurrg] ^iGufgpds.rrGsr tensorflow 
Q&UJGVUmf(f)f fllXl_fhl&6S>GfTU UUJGSTU(f)f^U Gurr&lGroTTLD. 
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30 features 


wl = 30 columns of random 



https://sist.sithub.com/nithvadurai87/88a6f4e571200ce75del7464f9fl3941 


from sklearn.model_selection import train_test_split 

from sklearn.datasets import load_breast_cancer 

import tensorflow as tf 

import numpy as np 

import matplotlib 

import matplotlib.pyplot as pit 

from matplotlib.colors import ListedColormap 

def normalize(data): 

col_max = np.max(data, axis = 0) 
col_min = np.min(data, axis = 0) 

return np.divide(data - col_min, col_max - col_min) 

(X_cancer, y_cancer) = load_breast_cancer(return_X_y = True) 

cmap = matplotlib.colors.ListedColormap(['black','yellow']) 
pit.figure() 

pit.title('Non-linearly separable classes') 

pit.scatter(X_cancer[:,0], X_cancer[:,1], c=y_cancer, marker= 'o', s=50, 
cmap=cmap, alpha =0.5 ) 
pit.show() 

plt.savefig('figl.png', bbox_inches='tight') 

X_train, X_test, Y_train, Y_test = train_test_split(X_cancer, y_cancer, 
random_state = 25) 

X_train = normalize(X_train).T 


52 





Y_train = Y_train.reshape(l, len(Y_train)) 

X_test = normalize(X_test).T 

Y_test = Y_test.reshape(l, len(Y_test)) 

num_features = X_train.shape[0] 

X = tf.placeholder(dtype = tf.float64, shape = ([num_features,None])) 

Y = tf.placeholder(dtype = tf.float64, shape = ([l,None])) 

W1 = tf.Variable(initial_value=tf.random_normal([8,num_features], dtype = 
tf.float64) * 0.01) 

bl = tf.Variable(initial_value=tf.zeros([8,1], dtype=tf.float64)) 

W2 = tf.Variable(initial_value=tf.random_normal([1,8], dtype=tf.float64) * 0.01) 
b2 = tf.Variable(initial_value=tf.zeros([1,1], dtype=tf.float64)) 

Z1 = tf.matmul(Wl,X) + bl 
A1 = tf.nn.relu(Zl) 

Z2 = tf.matmul(W2,Al) + b2 

cost = 

tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=Z2,labels=Y)) 
train_net = tf.train.GradientDescentOptimizer(0.2).minimize(cost) 

init = tf.global_variables_initializer() 
with tf.Session() as sess: 
sess.run(init) 
for i in range(5000): 

c = sess.run([train_net, cost], feed_dict={X: X_train, Y: Y_train})[l] 
if i % 1000 == 0: 

print ("cost after %d epoch:"%i) 
print(c) 

correct_prediction = tf.equal(tf.round(tf.sigmoid(Z2)), Y) 
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) 
print("Accuracy for training data:", accuracy.eval({X: X_train, Y: Y_train})) 
print("Accuracy for test data:", accuracy.eval({X: X_test, Y: Y_test})) 


ffiijGVid&mssr <sS\endaiL: 


Input Layer-®)placeholders-g> a_0«u/r<S(g^su: 
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uu5)jD@d(3j ^erfl^^jerrerr grjeyaefr training, testing eresrigj iSlfldauuif®, normalize ldjdqjlL reshape 
Gl&iuiuuuQdhssTfDGST. ifesresrrr, ^_errerfif® lcjd^jld Qeverflufif^d, ^rrerjaesrerr rffu^rrsb 
QrEL-Qsvrrijd^d Q&eviddl uudljodl ^erfluu^iDarresrX, Y eresuid 2 placeholders 
^(r^Gvrrdauu®Slssrpesr. (ffeurnffilesr <sv Lfihu frapp rresr ppeijaerr ^esresrp&fi id rflu^peb 
GlrEif-Qevrrijd^d Qaebdlesr/Desr. ^evjDffileb ^esvjvLjid columns-esr eressressfldesra LDrrdiliflg 
prjeyaerfleb ^errerrfeatures-esr eressressfldesrauSleb £g) 0 <s<s (Severn® ld eresru^foarra 
X_train.shape[0] eresruajiid, rows-esr eressressfldesrauSleb [Err ld (^rtfuurra erevevenei) <5/761/ 
LDrr^flaesrerr Qaevidau QurrdlGforrid eresru^j Q^rfliurr^ uiXaf^Usb None eresru^id X-esr 
evLpevesuDuiSleb G)arr®dauuLl®errenesr. ^eiievrrQjD Y-esr evLyevesunuiSUsb, eyGp ey (retarget 
column-eb <S4esvjvLjid uebGevtpj records eresruesr^d (^ffldt^id svesrauSleb [l,None] eresru^j 
Qarr®dauuLl®errerr§]. 

X = tf.placeholder(dtype = tf.float64, shape = ([hum ^features, None])) 

Y= tf.placeholder(dtype = tffloat64, shape = ([l,None])) 

^uGurr&n flg^peb QrEL-QevrrijddlesT (Lpgeb <s>K®d®eb Xtrain-eb err err 30 features-darrest 

placeholder nodes ^esnrirE^lQ^d^id. 


Hidden Layer- da near ^erre^^daesxsrr ^esiLDggeb: 

placeholders GVL^lGuj^rresr epeiiQevrrqj) feature-eb 2 _erretr ^psrja^id layer-d(^d 

QaeoidauuCfnd. r31®d>9> layer a,mssr hidden layer. ji shallow neural network eresru^rreb epGrr eyqrj 
hidden layer-grresr $i(rijd(&jid. ^evioffleb ^esvjvLjid nodes-esr erememfldem&emu rsfrid 8 eresr 
evetsrpujguf^jerrGerTmd. eresrGev £g)/ 5<5 8 nodes-dan esr <s 4 errerf(njd&,err (weights & bias) Wl, bl 
enssmd Quiufleb ^esuDrE^eherresr. 

Wl erestiid variable- d(3) tfrandom_normal() Qpsoid 30 features-da rresr ^]m<5i](ri)dam epeijQevrrp^ 
node-d(&jid gesflggesfliurra random (Lpemroufeb G^ijif(o)^®dauuLd® agenda LD®luurra 
^esuDdlfo^]. ^t^rrev^i hidden layer-eb s_errerr eyevQevrrq^ node-id random- ^a 
G^ijrEQ^®dauuLlL- 30 weightsparameters-go£ prhidluSlQTjdigid. eresrGsv 30 columns-da mssr 8 
rows-god Glamsmi _ s ^0 Qi^esrarrij ^j^leb 3-(rrj<5vrrdauu(®id. 

bl lores) 1 id variable-d(a, ajevda Ld^luurra ^gogShuid eresruan ^es) uvLjid. hidden layer-eb err err 
eyevQevnqij node-d(apd ^pesvssr ^esuxida tf.zeros() Qpevid eyGrj column-eb 8 LjgrgShurhiaesrerTd 
(olamsssri— epqr) Qi^esrarrij ^(r^6vrrdauu®SlfD§]. 
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W1 = tf.Variable(initial_value=tf.random_normal([8,numJeatures], dtype = tf.float64) * 0.01) 
bl = tf.Variable(initial_value=tf.zeros([8,l], dtype=tffloat64)) 


Output Layer-rresr <£/syrffl/0<s<ss5>eYr ^aoio^sb: 

a<ss)L—@hurTai s^shm^j GlsusiflufLl® layer. ^msiprFjd&m W2, b2 ST&oiih Quiuflsb 

<S46$)LDd3iuu®£hsiTin6$T. Qsusiflud lL® layer srsiru§] syGp syq^ node-ggd Q&msmi—gi. 

(LprB<ss)^UJ hidden layer-sb 2 _shsn8 nodes LD^luipsiGm ^^jD&rrssr features. ^gGsu 

®j[5& 8 features- d&rrssr ga 0 node-spstoi^uj ^snsyqijd&sh tf.random _normal([ 1,8]) Qpsoif 
su sin rr uj try d&uu(fS sir /bsst. ^susurrGjo <S4[5& «£0 node-spooL^iu bias ld^Iul/iI) y®g®5)uj if 
tfzeros([l,l] <57 sst su&opujfpjdsiuuSSfD^]. W2 = tf.Variable(initial_value=tfrandom_normal(]l,8], 
dtype=tf.float64) *0.01) 

b2 = tf.Variable(initial_value=f.zeros([l,1], dtype=tf.float64)) 


qpsirGssmdSu urjsijpsb (ForwardPropagation): 

hidden layer-sb ssbstT 8 rflufpmssTaish placeholders- sul^IGuj suqjfSsirtn <5/7Si/<S(6Y5/_<oOT weights 
LDfDgjLC) bias-god G&rjggi pssrgi 3isssrdd(fi3iss)sn^ ^jsusurr^j 3iessrdSLL(f\d 

Ss$)l_f^ LD^IuGu Z1 6TS&TIQJ ^QS)Lpd<3iUU(f)S/D^]. ^]SU/f)StS)/D 0 <3]<sb(5V&i] 1 <oT6S)ILC> SUQS)S,uf<SST 
dLp asssflda sigmoid- d(3) uSylsorr&i relu <S 4 sbso§] tanh GurrssTjB STgrrsuGlgrrsirstninu 
ULudrutfif^somh. hidden layer-sb 2_shsfT 8 fluppmssrs.^Lb ^giGurrssTjn 8 susnauurTssr 
GsssfluLiasnsrT G) su sifl uSl(f) Si sir jdsst. (gfigiGsu A1 §/i >. ^LDLD^IULii^esr output layer-sb s^shsrr 1 

rflu^prTssmssT&j ^ftSsnsmi—UJ weights LD/D/rpif bias-god G&df&j Z2-god s.smdSSSrogj, 
^ILDLD^Iuss)U iSsm/fif 0/1 -sir dip susm&uuSf^f Ggsmsuidlsbsmso. Tensorflow-go u 
QurT(n)t 5 t 5 sus$)rr as$)i_d) layer-sSl(r^if gj Gsusuflsucj^if activate Q&iuuJuui_rT£ ld^Iu&ouGuj 
^jLpuss)ud asmdSSsugjDOirTssrcostfunction-dtGjGhGl&spigg GsusmQnf. idssrssrrj cost 
function ^ssrd^shsnrrsiGsu sigmoid (gpsoLD 0/1 srssr activate Q&iugrfsdKfiLb. 

Z1 = tf.matmul(Wl,X) + bl 

A1 = tf.nn.relu(Zl) 

Z2 = tf.matmul(W2,Al) + b2 
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$)Lf)uemjd «C55T«©®^su (Finding cost): 

sigmoid_cross entropy() GTSSTUg] Tensorflow-<50 (sfjLpU<5V>ud ansm^lSlL^dau LlUJ6STU($)&>)(Q3)l. 

(g£lpjD(3j<sh &>65)i _ HfI layer-sflqjjrB&n GhsusiflsuQ^dUssrp activate Q&iuujuui—np Z2 onsssfluLian^if, 

^smstDLDiurrssr Y LD^luLj3i^fT,Lh Q^svidpuuCfSlssrpssr. <sujD6S)fD esxsup^jd asm® iSl Lpfp 
LpuiSlssr &jjrr&files)iud tf.reduce_mean() uiusiruQdlingi. 

cost = tf.reduce_mean(tf.nn.sigmoid_crossentropyywith logits (logits=Z2, labels=Y)) 


Gradient descent opsoih f^jipuanud (9j<ss)jDt5t5<sb: 

Tensorflow-eb derivative-god asmddl lL®, < 3 ]p®snjyuu®ni_uf<sb ^snsi^Q^daissxsn Lnrrrdrfl, 
^jLpuiSl&sxssrd (&j<ss)jouu<s$)£ spGp sufluflsc Q&iug] aS)i_<somh. GradientDescentOptimizer() OTgn/fi) 
function ^ppamssr Gsuesxsoevnud Q&iudljDgi]. 0.2 <oTgstu§] {g£l&fD(8j ^sdldsajuLLfflshm learning 
rate ^ 0 /A Q&tssip ui^uSlsc 3i<sm(^iiSlu\.pp cost lo^Iul) ^ipssr lB§] Q&ujgvuQlc) minimize() 

6TG$)iL£> function-d(&j<sn Q^eyidpuuCfSlp^t. 

trainnet = tf train. GradientDescentOptimizer(0.2).minimize(cost) 


Session-go S-(^eurrdSl ^esxststpGSt^tLjih ^jiud^peb: 

tf.global_variables_initializer() <oTGSTU3tf r5LD(Lp<ss)i—UJ r£lrr<sfl<sv rBirif variable gtgst <suss)pujpip§i<sfTsn 
^ss)esrpes)pin ld, ^ipssrpssrgisuds, LD§2una<s$)6fT Quppjd Qsimsrrerrd Q^iinLfih. s ^0 session-go 
a_ 0 Si 7 rrddl, ^pssr ^Lpsoif function-go rsml) {^jiuda GgussstQld. LShssnssrtjfor loop ^pscnf 5000 

&Tfdpj^(5ts)m ^(j^surrdd}, &fnn9<spiLC> cost <9ism(f)iSlLpuup/D<3irrssr rflrjssxsou-iLD, gradient 

descent Qfxscih ^empd (^stopuuproaimssi jflpssxsoiLjLb ^jiudsi Gsusm(fif.^susurrGp X, Y- 
d&rrGsrplaceholders-go GUS&rjujpjpgiGrrGrT Xjrain LDjnpjLD Y train -si) 2_syrsyr<$/7Si/<95S3>S)T 

feed dict = {} (gpcsouma iprrif Q&svidtdliLjmGmrTil). ^pprjGij&tisrijdamssT cost, 5000 (Lpoop 
3iism(^)LSlL^d3iuuLl(f) spdjQtsunqij (LpestpiLjiii gradient descent ^Lpscif (^es)pd3iuu(^d}p§i. 
(&j<ss)jnd<9iu u ifi— LpuLi c 6TSS)i LD variable-<sb [GD, cost] stsst Q&rrQd&uuidQmmpfdtgf) ippu 

[None,0.6931285163990998] STgn/fi) (Lp&opuShsv G&LSldauuQdljngji. ^j^l<sv suscuudsub ^errerr 
cost-go ldlL®lc> G&iflda sess.run()[l] stsst Q&rTQdauuLLQsrrsrT&ii. 1000 

&fDf!)]a(<sn)d(3) <sp(ij)(LpGS)fD cost-go iSlflssknl Q^iuiLjLDrrpj if condition (gpsoil) 3^fSlu]<shG<snmh. 
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S-Q^eurTdSliqefrenT rffgj,q<ob QrBLLQeurrijiss >« wSZiuiS® Q&iugeb: 

< nisi]Guerre ly ansssfluLiansrr Gifliurra fl^Lprs^ierrerresr <arsiru^rD3imssr GlL-Ssr&rTij correctprediction 
STgo/LP Gluiuflsb ^Qij6iirTd&uuLL®<sh<snaii. ^stsxsusi&flsir ^rrrr^fldsirrssT Qi—ssr&rrij accuracy 
<57go/fi) Quiuflsb 2_0Si//Td5d5/j/j/_l(5)siTS)T^/. ^i^ssi lB§] Q&uj <sb u (fi lc eval() function, uuffD&ld, 

&)rrsijfEs$) sytU- i ld, G&rTgGsxssrp ppsijaiGnGfTui ld palceholders-d&rresT i—gSilsb Q&svid&l. ^fpsmipssr 

accuracy-go u_jLb 8i<sm(f)iSh^ddliD§]. 


rfiijepi darrein QeueflufCK): 


Non-linearly separable classes 



10 15 20 25 


cost after 0 epoch: 
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0.6931285163990998 


cost after 1000 epoch: 
0.051629796747703585 


cost after 4000 epoch: 

0.0307175769807556 

Accuracy for training data: 0.9906103 

Accuracy for test data: 0.91608393 
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8 Deep Neural Networks 


spsir^jd^id QldidulIl. hidden layers-go ^(r^surrddld <s/D 0 /i> QrBLlQsurrrjd deep rSlu^rrsb 
Qrsi-LGhsijrTrjd ^<svsv§] multi-layer rflu^psb QrBL-Qsurrrjd istsst^j <s>l6S)Lpdauu (fid}jog]. Shallow-so 
rsml) sjfDQ&ssrGsiJ uuj<s$tu($)£§!hu srQggjd&rTLL® suLfikurrs 5 ^uGurrgj g^snpd < s / d < 3 l // tld . 

Q&sstjd <oT®£§ida;rTLlLq-6b pmh uiussru(^)^^luj ^GSxssTg&n rglrjssxsviLiLb £g)/i/ 0 /i> 

uujssTu^^^nLjshGmmh.. syshQsurrq^ Gseuuq^d^LDrrssr ^ensifq^danssxsn sas^puj^jd^ih 
(j})i—dd]6ViL£). ^ssxsn GGSsfluLi&esxsrT fi&Lppgiih (%)i—dd]6ViLi) ldlLQlci rflrrsv 
efilgSilujrT&uuQIdJfD&i]. 


wl — 30 columns of random w2 = 8 column of random 



https://sist.sithub.com/nithvadurai87/9d931f6b833bce58d6c2b0bdeedd266a 


import numpy as np 
import tensorflow as tf 

from sklearn.datasets import load_breast_cancer 
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def normalize(data): 

col_max = np.max(data, axis = 0) 
col_min = np.min(data, axis = 0) 

return np.divide(data - col_min, col_max - col_min) 

(X_cancer, y_cancer) = load_breast_cancer(return_X_y = True) 

X_train, X_test, Y_train, Y_test = train_test_split(X_cancer, y_cancer, 
random_state = 25) 

X_train = normalize(X_train).T 

Y_train = Y_train.reshape(l, len(Y_train)) 

X_test = normalize(X_test).T 

Y_test = Y_test.reshape(1, len(Y_test)) 

X = tf.placeholder(dtype = tf.float64, shape = ([X_train.shape[0],None])) 

Y = tf.placeholder(dtype = tf.float64, shape = ([l,None])) 

layer_dims = [X_train.shape[0], 8,8,1] 
parameters = {} 

for i in range(l,len(layer_dims)): 
parameters['W' + str(i)] = 

tf.Variable(initial_value=tf.random_normal([layer_dims[i], layer_dims[i-1]], 
dtype=tf.float64)* 0.01) 
parameters['b' + str(i)] = 

tf.Variable(initial_value=tf.zeros([layer_dims[i],1],dtype=tf.float64) * 0.01) 

A = X 

L = int(len(parameters)/2) 
for i in range(l,L): 

A_prev = A 

Z = tf.add(tf.matmul(parameters['W' + str(i)], A_prev), parameters['b' + 
str(i)]) 

A = tf.nn.relu(Z) 

Z_final = tf.add(tf.matmul(parameters['W' + str(L)], A), parameters['b' + 
str(L)]) 

cost = 

tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=Z_final,labels=Y)) 

GD = tf.train.GradientDescentOptimizer(0.1).minimize(cost) 

with tf.Session() as sess: 

sess.run(tf.global_variables_initializer() ) 
for i in range(5000): 

c = sess.run([GD, cost], feed_dict={X: X_train, Y: Y_train})[l] 
if i % 1000 == 0: 
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print ("cost after %d epoch:"%i) 
print(c) 

correct_prediction = tf.equal(tf.round(tf.sigmoid(Z_final)), Y) 
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) 
print("Accuracy for training data:", accuracy.eval({X: X_train, Y: Y_train})) 
print("Accuracy for test data:", accuracy.eval({X: X_test, Y: Y_test})) 


r&7<spidsirT6sr siSletrdaid: 

Gui/oasmi— sr ($)£§] da mhq.sb uiu 6 stu®£$ 5 Iuj umfuau LjiD^jGrBmudarrssr umljglflp ^rjspash 
train test split ^Lpsmd iSlfdauuffsugjjid, iSlssrssrf <S8®nsu normalize Qaiuiuuuidtf) reshape 
Qaiuiuuufflsijgjiid, ^shsufidff ^pspassisn fl^psb Qr 5 if-Qsurrfd(&jd(&jd Qaspijdajisuajiid, cost 
asm(f\i3is\.uu§j, gradient descent Qpsmd (3jss)fD[5& cost asmi—ifilsuaj], astOL^dliurra accuracy 
asmffiSlL^ijuaj GurrsiriD ^ssissrfa^ £g) i_ifjasiflspiid shallow ld/d/qjld deep rflg^psb Qir>ifQsurrfd 
spGrr LcrrShfiurraGsu Q&iusbuQdljngi. ^ssrrrsb iSlsirsuQTjid spQjjdlso ^ji^rfiasiflsb Lmdffud 
srSl^^nurrauu(fidlrD§i. 

1. Shallow LDjDQlid deep r£}uj,rjsb GlrBif-Qsurrfddlsb (Lpgsorrsuaj] Gsoiuf sshsifif(f)darTssr^rras^id, 
<S63>/_£) G>smurf Qsusiflufif(f)darrssr^rraspid ssiLmLjid. ^ssrnsb ^jsmi—uflsb 2 -shsrrhidden 

Gsoiussrpu QurrrQjpgi rjsm®id LDir^ju(f)id. shallow-si) spGp sp(r^ hidden Gsoiuf unlffud 
arrsmuuQid. deep-sb s^esr^jd^id Gld/bulIl- Gsmufash ^smorB^lQpdi^id. srssrGsu deep-sb 
spsiiQsiirTqjj Gsoiuiflspud s^shsrrnodes-darrssr ^snsi^Q^dassrsn sussrpiu^juu^rDtO) (Lpssrstsrf 
QiDrrf^id 6T3>3>63)637 <3 solurfaeh a_ shsrrssr? ^sarorflsb srppssrssr nodes s^shsnssr? stsstuss)^ rBrnd 
sussrijiutQjda GsusssrQid. ^j^roarra £g)/57(3>, layerdims = [X_train.shape[0],8,8,1] 67637 
QamfldauuifQlshsrr^]. (Lpatsorrsuaj Gsoiuf s^shsrfifQidarrssraj STshua,rrsb, uuflrodla, 
prjsyasiflsb ^shsrrfeatures-sir srsmsssfldstnastmu ^sfuu^rdarra rfiQjj X_train.shape[0] 67637 

Qarrffdaij u ifQshsfr&j]. assu_dliurra 2 -shsrr 1 stsstu^i QsusifhifLLQf) Gsoiuf so ^_shstr nodes-sir 
srsmsssfld 63)35 £g)63 Ti—uflsb 2 _shsfT 8,8 srsiruG^ ^jpsssr(f) hidden Gsmuf ash 

3— (rjjsorrauGurrsus<s)a>iLjid, sysiiQsurrQp Gsmuflsolid 8 nodes ^stowrB^lq^uusio^iLjid (^ffddlfDoj. 

layer dims = [Xtrain. shape[0], 8,8, l] 


2. .jy(/>!$<$<$/735 Shallow-sb spGp spQJj hidden Gsmuflsb 2 _shsrr 8 nodes-darrssr ^ sttsli (jpda&osfr wl, bl 
srspud Quiuiflsoud. QsusifhifLdtjf) Gsmuflsb s_shsn I node-darrssr ^sfrsrjQpdaoosfr w2, b2 srspud 
Quiufsond ^smwfG^md. gjoGurraji deep-sb (Lpa, so hidden GsoiuQTjdarra wl, bl srspiid 
Quiufsond. ^psmi—rrsij§i hidden GsoiuQTjdarra w2, b2 srspnd Quiuflspiib, QsusfufuQ) 
GsoiuQTjdarra w3, b3 srspnd Quiufsyid ^srrsiiQpdasiosirr ^&snodauGurrdlGromd. (ffifa, 6 
^SHSfrsrjQpdaiiSfpid for loop Qpsoid ^Q^surrdauuLi(f) parameters srspnd Qluiufsb dictionary 
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suipstflsb G&L&dauuQdls&TfDSST. ^j®<sb wl stsstu^j ^jrrsmi^msu^] GsmuQTjdarra 
<su<ss)pujrr)]d&uu(l)i<5u§i. (Lfi^scmsu^j GsmuflsSlQTjTB&i] suqjjiD 30 features- darrein 8 rows-god 
QarresmpQTjdt&jLb. ^eueunGp ,j>/(£)<$<$ Gemufleb a_ ehenw2 , w3 ^dhussxsu <s> 1 gfn(&j 

(Lpff)<ss)3)Uj Gemufle^Q^TB^i si/0/i) iD^ULidar^darresr ^ene^q^daemen 
QarTsmip(r^d(a ) iD. srstsrG su prresr layer dims[i], layer dims[i-1] eresr QarT®dauuif®shsrT^]. 
layer_dims[i] STesrugi weights ^essflufsb s^eherr row-esr id®uss)uiliid, layer dims [i-1] eresrugi 
^essflufleb g-snsn column- <sst ld® ues) u iq ld (^rflddlro^i. (Lpgeorreijg] Gemuqijdt&j erd,^ syq^ 

<31GrrsiiQ 5 ld® u em u iq id euesirnurrpdajB Gasmen ufsb so rr^^ rrsb,for loop-^esr&j 1 qp^eb 
len(layer dims) suesip <3fsm id suetD^d arremeorTiD. 

parameters = {} 

for i in range] 1 ,len(layer dims)): 

parameters[W + str(i)] = tf.Variable(initial_value=tf.random_normal([layer_dims[i], layer_dims[i-l]], 
dtype=tf.float64)* 0.01) 

parameters['b' + str(i)] = tf.Variable(initialjyalue=tf.zeros([layer_dims[i],l],dtype=tf.float64) * 0.01) 


3. Shallow-eb s^erren e^Gp hidden Gsmurf pesrd(3) (LpfstD^uj GemufleSlQ^TB^i si/ 0 /i> features 
iD®uss)uiqiD weights iD®ues)uiqiD G&pfgj Zl, A1 st 6$) i ld ld® u lSJ gs) ssrd aeSSfdSl®LD. 
ae$u_dlufl<sb ^errerr Gleueiflufif® G<soujitZ2 stqdild LD®uiSies)esr ldlL®id aemddhf® activate 
Q&iuiurTLDsb cost-d(&,d QaevidaiiD srssrrrp sprnQs.ssrGsu urrpfG^rriD. deep fiu^peb QrpifQsarrpd 
srssTfQ] GUQTjiDGuTT&i] qp^eb hidden GsviuqjjdarrssT Z, A iD®ULjaes)emqiD, ^pemi—rreiiaj] hidden 
Gemuq^darresr Z, A id® u iq asm emq id, aesiL^d) Qsusifhifif® Gemuq^darresr Z LD®LuSles)esnqiD 
asmddlL- GsusmQuD. ^^farra for loop syssrrrp Qaiuebuif® qpgesr qp^eSiebA _prev stsdud 
variable-eb X-eb 2 _ehmfeatures-esr LD®uiq ^smiDdauuil® qpgeb Gsviuqjjd&rrasT Z, A 
LDfsluLiash a6ssrdd)i—UU(®d)6STfD6ST. iSlssresrp qp&,sb Gemup aernddhli— Z ld®uGu A_prev 
iD®uurra ^ss)LD®rD§t. £g) 6 a><$ features-<^a QDsafgj GemuqrjdarresT Z, A LD®ULjaeh 

aemddl i_u u ®dleirineisT. ae$u_d) GemuQjjdarresT Z LD®uiq loop qpiprf,^ iflssr 
aemdd)i_uu(®dlrn§i. 

A =X 

L = int(len(parameters)/2) 
for i in range] 1,L): 
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A _prev = A 


Z = tf.add(tf.matmul(parameters['W' + str(i)], A_prev), parameters[V + str(i)]) 
A = tf.nn.relu(Z) 

Z Jinal = tf.add(tf.matmul(parameters['W' + str(L)], A), parameters['b' + str(L)]) 

[&ijepid&rT6ST QeuefiUdCfi: 

cost after 0 epoch: 

0.6931471575913913 
cost after 1000 epoch: 

0.6641999475918181 
cost after 2000 epoch: 

0.6641800975355493 
cost after 3000 epoch: 

0.2876489563036946 
cost after 4000 epoch: 

0.04967108208362405 

Accuracy for training data: 0.9906103 

Accuracy for test data: 0.91608393 
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9 Feed forward neural networks 


g-sheif®, QsuGifhtf® LDfinyib uebGeony ld«d/d((/)<s ^Qd^&iasxsnu Quid fry, 

spsijQGurrq^ ^Cfddlsonb ^ms^svrrssT rflu^fjrTGsr&Gsxsnu Glufiny efilerrihj^ib QfF,iQQ<sumjd 
feed forward filg^yeb QrBifQsorrfjd torssrny ^6S)Lpd3iuu(fi)d}fD§]. GiDfi&smu deep layer-eb it, mb 

&>6$STl _ 3jl $I&JD(3) <sp(jj [56060 6T(fiffi§]d<3HTLburT3i ^6S)UOLjlb. 2_GfTGfTlQ(Q) ^(filddlSST ^LfiSOLD 

Q& 60 ld&>UU(h)Lb [ 5 [J<SH&i<sh ^{5(0(3) <£l(ji)d(3j3i6lfl6iT 60 [flGuJ Q&6b<S0ILbGurT3,l. 

^[6S)6uprocess Q&ilhuuulQ® aesu—ld) ^Qddleb KesnstniemuiLi aossfluiSlsmsisT GhsueifluSlQIdlfDg]. 
£g) Lb(Lp 6 S)fDuSl 60 pjjey&eh ^etnQsrfiayib (LpssrGssrrrddl ldlLQGld Qpsoid^uuCfdlssirnssT. 

iS)<o5T6$)fTiQ-i ib (feedback) 6T6$ri d epesresijo 6T[5$ @0 s y(f)d(3jib 60Lp!hi(9j60$fil6b&s)60. ^pneog] 

<5/76i/d563>6)TLy QufinydQarTemu filg^pneir, ^rrsii&Ginm suLpdidlm rfluj } rrmss)idr3j iSlssrsyifudud 
Ql&iudl ^6S)iuLj6U3ii GurresrjD rfi&Lpsij spsyib rsemuQu/Derblebemeo. mesiGeo^nesr £g)g/ 

feed forward neural network meirny ^6S)Lpdauu(f)d)rD3jl. 

aemi—ldl GleoerrhifLL® ^(flddleb (ffgy aessflfig LDfiluiSlfiy^ib, ^smominurrstsT LD^iuiSlfif^LDrrssr 
Geonyurr® ^fiis.LDrrs, ^(r^d^ib uiQ&fifilsv ^i3,ss)iss)i_iu gradient (gpeoib &fitry iSlssrGssTrrddlu 
uiuessflfisy syshQsurrq^ ^iCfidSlsonb ^rfiQ^ssrGeo uiueZrufijififfiluj ^msiKj^dsi&om LDrrfifil 
<3t6$nr>d(3jib rfl&LpGii back propagation 6T<ssTfry ^&OLpd&iuu(fidlfD§]. iSlssrssrfj LDrrfifQUULdu 
^6fT6if(njd3 : >{6fi)d(& ) pysyaerr ^etnesTfigyib (LpssiGosrrrddld Gl&esrny l£ 6m (find &6s$fluiSl6S)6sr 
filaLpfiaydlesrrDesr. ^ebeorrGfD ffnn Gl&ujebuQdJfD&y. ^j^sb <sy6uQl6im(nj ^Cfiddlsonb 
fflu^pneZraiefileZr msm&ssfld&DSi (^filuiShbu ^snsLjd^d 0 S 3 >/dsi//t<s £g) 0 / 5 <s/T<sy under-fitting 
6T6ST/D iSlrjd&eisxssnuLb, ^msiid^ ^^ts.iDrrs, ^q^^rreb over-fitting 6 tgstjd iSlrjd&esxssnuLb 
6jfiu(fidlfDgy. over-fitting 6T6srfD lSI fjd&ss)6ST6S)ujd, fft^fjuu&.fi&rrs, sors&Gg drop-out otimization 
^(Gjlb. QlUIT3f]6UIT3> <3i$5l& <3]6fT61[ figj^gfTSSTSi65)6nd Q&fTSmU Q [BlLQ 60 fTfjSl6V ^j^6S)6STU 
uiu6sru(ffi^]6oG^ dl/Duurrai ^esmoLjih. <s>l£6sr accuracy-go ^filarrlda 2 _g 6 yib. ^emguufifil 
Regularization ld firry id optimization msvyib uq^filufeb (sftend&LDrT&d arremeomb. 

Different Classifiers: 

feed forward fig^rreb GrBifQleorrijdleir ^]6 S)uuul1u LC>6 S)fD(Lp& ^Qdqijaeifleb ReLU 6T6S)iib 
606S)3,UU(flfifiHLjLd, <S63>/_£) ^(fi)dSi60 sigmoid 6T65)ILD 606S)3yUu(Qififiluyib 
uuj6iru(fi)fi^uu(fi)dl6irfD6sr. 6T&,fi&,rr&, ^siiGonny Q 606 bG 60 fry Gossis.uuffiifi^ls.eh 
uiuesruQfipuuQdlesrfDesr? ^eofinydarresr Ggemeoaerr mesrssr 6T6sru6S)gu ufifilGtiuebeomb 
^)/57@ arremeomb. 
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sigmoid - £§)g/ gesrgi aassfluLfasnsn 0 (tpgsb 1 susntj ^iQS)Lod(^Lf. dLpdaemi— suonjjui—^^lsb x-eir 
ld^Iugs)uu Qurrrpjpgi aessfldauu^m h(x), 0-(Lpp<sb l-euesirr ^gsildiu (J!su6ssr($)QiD<ssfl<sb ^^parresT 

&LDG$runi rTGtsr&j l/(l+e**-x) OTSw/ry ^)0 <s@ld. sigmoid function-daman &LDesTum_rTa 

^]&s)LDdlrr)§]. 


^ Tanh - £§)j;/ pesrap assrfluLiaGsxsfT -1 (ippso 1 sussirj ^amrid^LCi. ^siisimp] ^fsmuuuppamsn 

&LD6srum—rTGsrgi[2/(l+e**-2x)]-l OTsir/ry ^(jpd^ih. assuSl Qeoediufif(f\ Qsoiufleb sigmoid ^sbsc/pj tanh 
Qumdrp ^frjosmpsb <ppmsuap spesrempu uiusnuf^ippsomi. 

^ ReLU - ^]3>sir assrfiuLj 0 <gf<sb<soap ^iLDLDSiluurraGeu ^smmiLjLb. Rectified Linear Unit GTSsruGp Relu 
€T<drtQGS)Lf>dauu(j)i&l[ogji. ^j^lsbx-ssr Ld^luq 0-d(&jd dip ^(njrsptreb, ssxssrO otsstsi / ld , ( 3 lo ® l > 
^ssuxippmsi) ^fiiiD^lui^sinsin ^uuipQiuiqLb asssflddlppi. gtsstGgi] priest ^jpsst autesrurr (J) max(0,x) 
<oT6st ^animdlpp]. 
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Sigmoid = 0 to 1 


tanh = -1 to 1 


ReLU = 0 to: 




Machine learning- s$t 3 iffil(Lp&,d &fDfD<sfl<sb G>susmQ) ld mssrmsb, Q su^jld sigmoid-gou uiuesru(gl^^l 
logistic regression, neural networks Gumssrrosurdss)JDd (ol&iugi urrijd&isorrLb. g^Gtsrmsv deep rfiu^jjsb 
QrBLlQsurrrjd <oresr^j su(r^idQurr^] ^jsm^Qiu ^svcssid,^ ^(^d^ansiflsb s^ehsrr 
r£lg^rrmdr&ifQfyd^Lb Q&ujisvuQ^gisiJgi &ffl<5iirrrT§]. sjQssresflsb LDsnjnQpa <3l®d(&j<3;Gifl<sb s_mm 
$ u^rrmsbr&sftlsbr lS§j back propagation gtssti d spsinstnin rsmb Q^iusbut^^^Gsumb. ^§1 
s^djQsunq^ 3 iCh)dd)<sviLb 2 _<sh<sn rflu^rjmssTasiflGsrderivative LD^Iues)ud a,<sm($n3L^d > & > i 
3j^ > fDGs,fDfDmjGurr<siifQ] weights-update Q&iuiLjih. 3jd&LDUJ^&>l<5V Qgu^j ld sigmoid-gou 
uujsiru(^^^l<ssrrT<sb, 3i§l Qsuqjld 0 ^sbsvgj 1 <oT<s$)ilL ld^Iuss)uGuj asssfluugnsb, x-sst 
LD$5luG$)ud gusssfl&LDrr&d (&j<ss)jnd<9i 2_<$6i//Tg/. (oTssrGsu^rrssr hidden layer-sb ldlLQld 0 ^sbsogi 
< 3 ir 5 px- 6 ST ld^Iugs)uGuj a<sssfldad3i*.Lq-UJ ReLU-esxsu it, mb u uj sir u(^)^§]d}GSTG tomb. a<ssu—!d) 
Geoiuflsb lolL®ld sigmoid uuj(diu(^i^uuLLQi<sh<sn§i. 
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10 Softmax neural networks 


Softmax <oTG5TU&ji multi-class classification-d(&j g-gsydlsirin suss)s,uu(f\f^l <^L,(3j ld. MNIST data 

i!>T<ssru&)jD(&j<oYT usbGsufpj sfilgihiasiflsb ssiaiurrsb srqpguuLLi— 0 (jpgsb 9 sussirj 3 ]i_rhidUu 
srsm&siflsir Qgrr@ULja<sh &rTGS$ruu®Lt). £g)g/ 0-9 srssnif 10 suss)& label-sir dfp 
3 lssuDiud^Lpuj 3isssfluLi3isS)srT filaLpgsjiLb. ^jsuinssipQuj multi-class classification-d 0 LDrr^lfl^ 
^/TSi/<95syr/T<95 rBirif uiusiru(f)f§)ld Q&rrshsnsvmf. ^surnnSlsb 55000 ^/rsi/dssrr uuSlffdl 
<gjSi?luugfD(3jLb, 10000 &)Ijgi]&>syt uuSlffd) Qujdjd Q^LfQsurrijss)ai G>&rr§iluupfD(SjLC) 
uiussruQppuuQdlsirrDSST. 

Softmax srsiru&fi s ^0 sSlsqtiuif QarrQd&uuLLi— spsfQsnrrq^ sussi&ufsir Sdqpif 
GsssfldauuQsijpiD&rTssr &rrf&>uud ^fpjs.ssisn 3 isrrd,^i ^ipjif. dipdasmi— srQggjd&mlipsb, 
softmaxQ STSsrug] «£0 srsm O-^s, 3 iss)LDSU£jD&rrssT &rrd,S>lujd^iQi srsususrrsi^, /-<-§/,<s 
3lssuDsu^ff)3irrssr &rrd,S>lujd^iQi sr sir sum sl/ srssru&j Quit sir pry GIldit^ld 10 sussia class- 
d^LDrrssr &rr£§i)uj<d3^f!)]a6S)sn sii(3j£aj]d Q&rrsvspiLf. £g)/ 5 <$ ^rr^^liuda^^jaisfr 0 (jppsb 1 suss)p 
<^ss)L£>r5& srsmasrrrrsb (SjrSld&uuQfif. ^)uu<5g/ LSlflsyasrrlsb <31ssurtrs&> STsms.ssismqif 
a^ifipssrrrsb 1 srssr snqrjif. CT@^*«fTLL/7« ssiaiurrsb 8 srssiiif srsm spshQsurrqrj 

sus$)&uflsir d(jpif 3isssfldsiuu(f)s)j^rd3irrssr LD^IUL/Lf, 3^ss)su3isiflsir Qprrssis; 1 srssr 

3 iss)LDSu^iii) iSlssrsoq^LDrrr^j 3 iss)LDiLjif. (ffssisu&siflsv sr&)sir ldS^ul] 3 i§h&i£>rT& ^(rrjddljDG&rr 
3f5$ guss)&u5)sst dip Q&rrQid&uuifL- srsm s.sss^ds.uuffif. 
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P(0) = 0.04 



https://gist, github. com/nithyadurai87/972397d9039d577a3be943779429f80 7 


import numpy as np 

import matplotlib.pyplot as pit 

import tensorflow as tf 

from tensorflow.examples.tutorials.mnist import input_data 
from random import randint 

mnist = input_data.read_data_sets("MNlST_data/data", one_hot=True) 

print (mnist.train.images.shape) 

print (mnist.train.labels.shape) 

print (mnist.test.images.shape) 

print (mnist.test.labels.shape) 

X = tf.placeholder(dtype = tf.float32, shape = (None,784), name='input') 
Y = tf.placeholder(dtype = tf.float32, shape = ([None,10])) 

W = tf.Variable(initial_value=tf.zeros([784, 10]), dtype = tf.float32) 
b = tf.Variable(initial_value=tf.zeros([10], dtype=tf.float32)) 

XX = tf.reshape(X, [-1, 784]) 

Z = tf.nn.softmax(tf.matmul(XX, W) + b, name="output") 
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cost = -tf.reduce_mean(Y * tf.log(Z)) * 1000.0 

GD = tf.train.GradientDescentOptimizer(0.005).minimize(cost) 
correct_prediction = tf.equal(tf.argmax(Z, 1),tf.argmax(Y, 1)) 
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 

with tf.Session() as sess: 

sess.run(tf.global_variables_initializer()) 
writer = 

tf.summary.FileWriter('log_mnist_softmax',graph=tf.get_default_graph()) 
for epoch in range(10): 

batch_count = int(mnist.train.num_examples/100) 
for i in range(batch_count): 

batch_x, batch_y = mnist.train.next_batch(100) 
c = sess.run([GD, cost], feed_dict={X: batch_x, Y: batch_y})[l] 
print ("Epoch: ", epoch) 

print ("Accuracy: ", accuracy.eval(feed_dict={X: mnist.test.images,Y: 
mnist.test.labels})) 
print ("done") 

num = randint(l, mnist.test.images.shape[l]) 
img = mnist.test.images[num] 

classification = sess.run(tf.argmax(Z, 1), feed_dict={X: [img]}) 
print('Neural Network predicted', classification[0]) 
print('Real label is:', np.argmax(mnist.test.labels[num])) 
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(55000, 784) 


(55000, 10) 

(10000, 784) 

( 10000 , 10 ) 

Epoch: 0 
Epoch: 1 
Epoch: 2 
Epoch: 3 
Epoch: 4 
Epoch: 5 
Epoch: 6 
Epoch: 7 
Epoch: 8 
Epoch: 9 
Accuracy: 0.9226 
done 

Neural Network predicted 3 
Real label is: 3 
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11 Building Effective Neural Networks 


6£0 flg^UGC GrBlfGlSUrTlfdlSSr 3iLLlSS)LCiULSlsS)SST SU^]SUrrd(gSUgJD(g (LfgsEsV SHfsb ST(LgSllSSTJD 
uecQsu^j <su<s$)3>iurnssr iSlrjd&ssissT&srT ujbjShLjib, s.isnjbssijDd aissisnsugrbig g-gsyib 
suflsuss)3i3ieh ujojEiuld Qgiflgg] Q&mshm GsusssTfib.. sp(g fu^ijsb Q tb EG) snrrijdl ssr efficiency 
srssrug] uuEifif^jsEdsiuuifisugifxg torsi]snsnsij Ggpib srfggidQsifTshdlfDg] srssrussiginib, 
Accuracy srssrugj uu5lfD/!)]6)EdauuLLi—Si]issT srsiisomsi] gjsbsEhuiJmad asssflddljng] 
srssrussigiLjib (gnSlddljog]. £g) sujdssijdu urrfdigiL &rTtjsstfl&sh ujofUqu), sbsujDesiJD srsiisurr^j 
assisrrsijgi srssrug] ujnjEu-iLb ^juu(gfuEsv syshQsurrssTjnrr&.u urrrrd&somb. 


11.1 Bias-Variance Problem 

grub ^-(j^surrddlLL/shsn QgEQsunijd uiiEjodhiEssr Gung] s> /£§)« shgttgii accuracy -go 
QsusEluuffdfssrrTsviLb. f offish sHgssmsb fljDLbuL- Q&ujsculL® &iflsurr asssflda 
(Lpi^iusElsossisoGhussflsc, sHg/btg bias ^sbsog] variance iSlrjd&ssissr flaLprBgjsrrsng] stsst^j 

umflflg pjjsiiasrrlsb ^srrsrrfeatures-sir srsssrsssfldssia grrsTi&sEssr srsmsssfldonasmu sfili— lEs, 
lE&d (3jss)fD<5iJrTa ^)(gd(gLbGurrg] bias srssjnb iSlrjd&ssissr srjDuQSilfDgi. srQssrssflsb spflrjsssrQi) 
features-go ssisugg] ldlEQGld 'Sufis, s]sns)Slsb SH ss) ld ffjg] sir syt g!jsn&>ss)GfTU u id fid sfog^jd 
Qsrrsnsn (LpiusbdlfDgi. ^fshsurrjnrrai s/didgc srssrug] urjsuscrrs, syssuiHumDsb, <sp/firrsm® features- 
gou Glurrnpjgg] ldlLQGld s^ssnrisu^] bias / under fitting stsstitj] SHSS)Lpd3>uu(f&l)jngi. 

S>iGg GunssTQ] umflfflg grjsij&siflsb 2 _shsnfeatures-sir srsmsssfldss)#, grjsyasiflssr 
srsmsssfld stS)SSS)UJ gEl_ lEs, lEs s^fs-Lorrs, ^j(gd(gib uif&gfsv, variance / over fitting stsstjd 
iS}ijd&ss)ssT sjjbuQdljDg]. ^jgfDsrrssr srrysmrdsmrrs ^rjsssr® sEls^iugssigd serose mb. 

(LpgsSlsb grjsy&sEssr srsmsssfldssisi (gss)jDSurTS, ^j(guugrrsb, s]sujdss)jd ssoufflsv 

ldsstu u m _ ld Qsujg]sEl(fidlfDg]. SH ®ggl features- sir srsssrsssfldssis; sufs s]sns)S)sb ^(guugrrsb, 

sjDQjLb SLburBgLEscsorrg features- uSh—ib ^)(g[5gl 3^i— sjGgrr spssrssijod &jDjr)]d GlarTsm® 
Slgssrsuifl Qsujgcul _ s^yibiSlddljogi. 

^jss)sos(sgds[TSST gussujui—lL dfdssmi^sorrr^j shs^ldilild. ^]ss)g srsrrluj giE if sc Machine 
Learning stsstjd Ljgg&gfsi) arrsssrsomb. $)ss)g s_(gsurrd(gsiJgjb'SrTSST flrrsvui) sjuLiggagflsb 
'PolynomialRegression’ stsstjd u(gfluEsb Qs.rrfds.uuEfshsng]. 
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£g)<95<$63 )<suj bias-variance lS) pd&GSxssTSSHud) ^rjd<9i apjGq&csxsfT rBrnd training, develop, testing <oT6$r 
ZLpGsrfDrrau iSl figged Qaarsnsnsomh. ^j^esr (Lpsoih rsml S-(j^<surrdQiLj<sh<sn umi—tsb <oT[ t,&> suenau 
iSltjd&<ss)<ssid(3) ^ strrrdlit/shsngi (orsiruGS)giLiLCi aism^iSlLpdaiscmi). training-daoresr gptsqanssxsrr 
ssxsugg] uuSlin&l ^eiflggn LDm_asxzo ^(f^Gurrd&GorrLh. develop-d&mssr grrsii&asvsn esxsuggjd 
Gl&msm® UfflG&rr&ildigl epQTjdlso hyper-parameters go Gsusmi^lUSUITQJ LDrTJDffi} ^SS)LDgg]d 
G)&msh<srT6orTL£>. LSlesresrtj testing-da mssr gqjGqa err QpsoLD, wni—etneo uflG&rrSilgg] ld^ulSI insert ld. 
^jissxsu <sp<sijQsurrsirr^jLi) Ghsueifluu(^)gg]Lb cost Lo^luLjaesxsn osxsudtgjd Qamsm® Lornsc errsp 
suesrau iSlrjd&G$xs$rd(&j <sd,snrrdliqeberraji eresruemgd s,emQ)iShqggi efih—eomd. 
’oTQtdgjid&rTLLi—tTa, 

1. Training error 80% -d(3jld ^dh&LDrrs, (°£] (Tt) [Eg ITSO bias iSUjd&QSXSST ereSTQJld, 

2. Training error (^eS)fD<SUrTa<sqid (2%), Dev error <3]§>]&LDrr<9iGyLC) (30%) ^ (Tlj [Eg ITSC variance iSlljd&QSXSST 
ST SSIfO ILD. 

3. Training LDfDfQ] id Dev (sfirTSmLgGVI id error LD^lUiq (3jGS)fD<5lJrrasqLD, Testing error LD^lLJiq 
^tBlaumaeqid ^(T^rEgrrev, variance iSlrrd&emesr training & dev (sfirTsmLy&viid rffaLprE^ierrengi eresr^j 
^qjggld. 


Early Stopping erssrug] ^juiSlrrd&os)ssrss)ujd, ^rfda a_^s qid 6^0 suL^lsuesia ^(^id. Normalization 
eresrug] bias iSlpdaemssrss) uj iq ld, Regularization ereirugfi variance lSI pdaesiesresnu iq id ^pda 
p^gsqdlpgj. weights decay, drop-out Gurresrpesxsu rflu^psv QrELdGhsunpdleb id id® id 
uiue$ru®gguu(®id %ul/ eu etna regularization jgiLdurhiaen ^(gjid. 
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11.1.1 Early Stopping 

Gradient Descent ^LpscLD error ld^IuL j &*.®<ss)a ^ss)i—UJ (LpiuspudiGurTSii. Training lojdqjlL Dev 
pern® lc> (ohsuGifluuQp&n&hssrfD cost ld^Iugs)u s ^0 sl/s!d/tl//_ld/t« susnprB&j urrpd&,spid. 
Training ld^Iuli (&j<ss)/Drsgj Q&msmGi— sarB^msv syG&rr® G&prsgj dev lo^ul/ld (GjsniDrBgJ 
Q«/tsw(5/_ sup Gsusm@LD. spgrrsuQgrrQij rfisnsoifilsb dev ld^Iul/ Qprri—ihiSilssrrreb, 

&LpfD&)6S)iu ^r5rffes)60uSlGGoGuj rflgjgtsIsfilLl® ^^ssr ^srTsiprFjd&sinsrT ipLogj iDrrL_GVid(&,u 
uuJSSTuQ)d,^d Q&rTGhmsvmJ). g^&jGsu 'Early Stopping' STSsrrrg ^sviLpd^uu^SlfOgJ- 
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^i^rrsuaj &*_®6S)3i (Convergence) <3]65)i—Gu^jf)(& ) (LpesresrGrr aLpjD@ rfl^j^uuLlff) ^sfrs^Qfdasrr 
Q&>[f[ 5 &!&)(]))&>&li u(j))si]3j it< sb £g)g/ £g )uGuiufleb <3l6S)Lpdauu(jfd)[D&i]. 


11 . 1.2 

11.1.3 Normalization 

speiiQeurrqij feature column-si) ssrrsrr iD^luLfai^ih QsnsiiQsn^i srsssr srsossisoastflsi) ^ssiLdrs^lQ^rB^rrsi), 
^6S)6tsrdss)£iLiLb-l sSlQTjrsgi+1 ffuovij, ^fsbso§! 0 s61(ijr5§i 1 Gusmrr stsst Srjrrd^suCS^ Normalization ^SOSO^] 
feature scaling stsstuu^Ild. lSI 65T6U(Tf} L£> (QdSfrrdsinpij utussruQdSil SpadauuiLi^rrsb ^§]Qsn mean 
normalization STSmjuQul). (ff6S)^U UJDtfl ^jSSTSfllLD Q^Slflsurra <3]f)IUJ S^QfLDlSlstSTrTSV, STStflUJ 
&>iff l£I60 Machine Learning STSSTIT) L/35£>d!>5>^D<5L) 'Multiple Linear Regression' 6T6Srro U(0)&)1 uSlsb 3UT6SST61ILD. 

particular value - mean of all values 


maximum - minimum 


11.1.4 LI, L2 Regularization 

iff a 5<s (&j6S)fD[5& < 3 jSfT 6 ii ^rj6ija^d(^, s ^/fflTsiy urhiasifluu^rrsb 6jrou(f\6uQ^ iSlrjd&emem 
<oT6srf£j untj^Q^rrif). ^)sa>^ tzeffrjda 5 spsirigj <s>l§fa; ^errsij ^rjeijasnend Garrsmtff surrsomf 
< 3 ] 6 Vsog] sp eii G) 611 rrqTj feature-sir urd a 6 i?Iu6S)uiliiJ) t^esifors^ ^srrsffsb iDrrfrQsorrif. (gfGfeb < 3 ]§fa 
^snsif ^rjG^aesisnd GarremQf euqTjSug] ifftajrp^ Gr5rj(Lpif) Gasosyif iSks^d^if) 

Gsuansoum^sorrsb spsuGsurrqr, feature-ssr urfiastflusnuiLitf (ganroda r3i&> go/ i _ sir regularization- 

darr&ST ^snsf(f ^fsmsmdauu(if dirndl. 

GrBnSluufff^LD (LpemrouirresTSfi LI LDfDfrpif L2 srenuf rfJansoasiflsb 

[ 5 esu—GurgjSil£)&,]. iff a iff a <s>l§fa; ^snsflsb features ^jQ^d^ifiQurT^] LI susma 
uuJGSTuQfgpuuGfdljDg]. (fflg] <3l&(a (tpddluJt£lsbsorra, features-darrein iDdfuiSletnesr aiJfuJLb (0) 
<oT6sr ^6 S)lduu^6st ^Lpsoif (LpddhuijmesrsufDesifD LDLLQfiJ) uihiaeiflda 6S)6udd}jD§]. sf(r]) regression 
urn iso rresr g/ LI-®ou uiussrutflf^lssrrrsb ^31 §] Lasso Regression (Least Absolute shrinkage & selection 
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operator) <oTSSTQ]ld, L2-gou uiUGSTuQptglGsmcsb ^§1 Ridge regression 6T6srrrgLb 
<3j6S)Lpdauu(fi)d)fD&i]. L2 <oT<sstu§i Q^ilQisurTijSlsb sshm ^<sn<sij(iijda<srrl(SST 3 ^lL(j))£ 

Q^mss)3iss)UJiLiLb ^piLi^uundSl ^i^ss)ssr sombi—rr/2m 6T6 S}ild LD^luiSlssmsb Qu(r^d(^dlfo§]. 
^susurrjDrrs, LI ldjd^j id L2 -^sb &6mi_f)lujuu(fi)id ld^IuQu 'Penalty term' ffTssr/ry 

<s>i65)Lpd&UU® Sting]. L5l6ST6$Tlj ^idlD^luGu cost (J) LD§>hjL]L _ 6ST ^6ti)6md&UU (§)&>) fDg], 

&(njdaijmad Q&rTebsouGurresrrTsb LI Greirugi 'Absolute value of Magnitude' 6T sir fry id, L2 6T6stu§] 
'Squared Magnitude' 6T sir fry id ^nShuuuQid. 


J = J + Reg-term 
Reg-term 



Lasso Ridge 


$]LC)(Lp6S)fD&6i?l6b it, mb 6U6S)piu^jd(^Lb somdi—rreiSleir ld^Iuli lS\& ifl&d 0 ot>/d<si//t<s £ 3 ) 0/5 < 5 / 7 <sw, 
overfitting 6T6 sti d iSI pd&enesr ^ uuiq.Giu fiJesrso^^Jq^d^id. i£la 5 u 5?<s ^^anDira ^(njfitzrreb 
underfitting filesisod^d Q&sirrrps/iKfiLb. 6T6 stG6u £g)/s.$ su/tldl/t in^JuiSlesiesr (3^/jsi/ 
Gl&iuiLjLbGurTg], i£l& iSl&d asiiesTuma £g)0<s<£ Gsosmugiu^] ^eudhuumdlfD^j. 


11.1.5 Drop-out Regularization 

£g)g/ s^si/Qsi//T 0 Ggvuj ifhsvi id G^rrpmuLDrrai epQTjdleo nodes-go fiddl sfiUfiso^sir Qpeoid features-eir 
STsm&ssfldstn&smud (gj6S)fDd(&jLb. keep_prob Gresrug] £g)g/ uiusiru(fifi^]id parameter <Sf(&,LD. 
^]3)6 $t g/)s vid spsiiQsamr^ Gsoiuiflspiid srsdsornsq nodes <gf\(njd&i Gsosmifiud sresrues)^ 
6U6S)piuQjdai6cmh. < 3 i^rrsiJgj keep_prob = 0.6 Gresrigj ^(r^rB^rrsb 60% nodes-go ssisufi&^d 
Q&rrem® idfsl 40% nodes-go fiddl sfihsomd 6T6$rprp ^yrfp&id. 6 £<si/Q<siv/T 0 Geoiuq^d^id 
QsusiiGsurru keep_prob i£>$gluLj&6S)6n rsmd Guesrtjujrryd&iGomd. 
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11.2 Data-lnsufficiency 

uuS)fD@ *s>i<si?luup(D(Sju Qutt^lottgst &<sh £g )ecsomsmLDUJrTGC GjjDuQdlGsrfD lS?/7**63)637 

<^,0/A ^i&j<s$)6fiT£5 poS)ijda Data Augmentation gtgsttd (Lp<ss)jn<s$)UJU uiuGSTuQ^Gomb. 

<5/761/* 63)6)7/9 (o)/_J0<*0<56i) GTGSTfQ] QurTQ^GTT. <S>1£ITG1J§1 TBLDlSU—LD 'SJIdQ&.SStQgU ^(J^ddlGSTfO 

<5/76i/*65>6)7* Qamsm® <j>/( 5 )* ^ens^leb < 5 / 761 /<*65)6/7 ^(fTjGurrddtu uiugstu^^gottlci. Mirroring 
(iSlrr^usSl^^svj, Random cropping (dpjbfD §]Gm®&Grmd(3)gGb), Rotation (3TLpib^J^<sv), Shearing 
(QgulL^^go), Colour shifting (r§l idld rrrnpii> Q&iupsb) Qumssrpssxsu £g)<5<565)*i/j <5/76i/*6yf?637 
Qu(ri)d&£§i}rD(gj ^-^g^SIgstjo (Lp^jD^sh 


11.3 Vanishing & Exploding gradient 

rff^psb QrBL-QsurTijSileb, TBrnb uuj<sstu(^\^^Hnshsn weights -gst ld^lili lSIshsi^ld dlnfliupna 
^j(FfjfE&)[T<oV, ^{£ GST gradients (dw, db) backpropagation -gst Gutt&j 6£0 *l/_ <5(5)61) < 5 / 7637 / 7 * 
ldgsi/T) fE<9}]G)5Uj)ilb. ^]^]Qgu vanishing gradients gtgstuuQlci. $)&JD( 3j (3/5/7 LDrrrQrrs, gradients-Gsr 
LoSilULi lSI&li Qurfliugrra ^gs)LC>gij§i exploding gradients gtgstuu®L£). RNN Quttgstjd /§ 6337 L_ 

G)3>ITl _/7L/63)/_ UJ 61//7/7<563)<5*65)6)7 /#63)6376)5) GO GnGUgSjjd Q*/76)T6)7 QGUGSSTipUJ 5/9*61/i£> ^LpLDITGST 

Qr 5 LbGlGlJrT!j( 3 jd(&j £g)<5<563)*t/J /_5)/7<**63)637*6)7 67(/_£>® GST ft) GST. ( RNN LlfDlfl £g)63$61/0ii> U(Sj$aGlflGO 

s,rTGSSTGonib) 
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hidden layer-sb tank activation fn-gou uiusiru(iff^isuo,siT ^tpeoih (gfiggsnauj iSlrjdoos)ssra ss) sad, 
^srSlfdasonid. Gld solid) Xavier initialization Qpsoid weights-darrssr gjsuda Ld^luLjasiDsn 
<3i<si?ld(&jil>(]>urTSfi, sVgji LD^JuLiaerr LSlasi^LD ^Ir^hu^rras^id ^jsbsomDsb, iflasi^Lb 
Gufliu^rras^id £g )sbsorriDsb ^ssnDsustng Goiudlrog]. ^grrsugi weights matrix-darrssr 

ojsuda iD^IULjassxsn ^srfld^LdGurr^i, sir variance = 2/n (where n = no. of features) srsirrrp 
^(rijdtajLimnpj umjf^ldQarrshdlin^]. ‘Gradient Clipping’ srsirug] exploding iSlrrdoss)ssrssuuf 
<5 sfilijda 2 _<$ 6 iyi£> s £0 suiflsussia ^(Ojid. £g)g/ gradients-sir LoShuLjash spqij (0)f£luidlidL_ 
srsbstnsossuug ^rrsmi^d Gosbspj ldGu rrg], ^suifissip srsbssysod^srr ^ssild^^i iSlsirssrij back 
propagation surflGiu Gospigdil weights iD^ULjostsisn G>LDidu(f)f§i!d)rD§]. 


11.4 Mini-batch Gradient descent 

Gradient descent srsirjDrrsb amusy suLbarrsusifl srsir^j Gurrq^sh. ga 0 GrBidGsumjd 
(g)Ld(Lp6S)jDGsnuu uiussruffd,^! a^(f)svta(convergence) < 3 ]ss)i_uj (LpiusoiidGurrai spsiiGsurrq^ 
OLprndhidlsviid. (Lp(L£>& ^/TSiyassxaruyii) uuasiru(f\d, {flu uudlfdl ^sifluugrrsb, <3i§)a ^gj/arsiy 
GrBP(LpLD GoiusodUnsniLb sfsmrrdlfo^]. (ff^s$)s$rf rjuu^roarra surs^G^ mini-batch gradient 

descent ^(3jLb. ££]§] (Lp(L£>& grjsyastnsmLiid) u so G surry G^rr^^lasrrrrau iSlfipg], spsuGsunqij 
ayrpidJudlsoiid spqjj batch-go ^synuiSl ^gssTL^uusmi—uSlsb gradients Ln^luLjassisn §)(njd,a,id 
Goiudlrogi. srQggjdarnli—rra uudlrodl^ gpsyasiflsir srsmsssfldssia GIldit^lL uggrrudlpid 
(gflQTjddlinGpssflsb, (gfigi < 3 t,udlrrid ^uSlpuma ^ssildib^ GiDrr^ih 10 G^rr^^Jassisrr 
a_ 0 Gi//T>* 0 /i). idlsirssrrj spsuGsurrqjj aip/pdldtajih spsiiGsurrqTj G^rr^^liurra ^spiuiSl gradients- 
go &)(njd,o,id GoiuiLjii). ^giGsu Stochastic Gradient descent srstsruuQni >. < 3 ig]Gsu iSlfldauuQid 
Gp, it 0 d)ta( dW ssr srsssrsssflassia G iij00,0,0, o,rj q ja( j) 7i(o)j <oTGfiftr6$tifla<sfi)aa(Oj OLDLDira Lio>ojrTu5l ijld srssr 
^issun^rrsb, ^ojGsu Batch Gradient descent ^sbsooj Gradient descent without mini-batch srsirrrp 
^ies)Lpdauu(^)dljD§]. 

Stochastic ldjd^jld Batch ^\<sijaSipsssr(f)Lh orriusy suidarrsusifl qpsold a*.(f)ss)a ^is$u_susv>o, 
(Convergence using gradient descent) sussipui—Lnrra sussipd,^! urrrff^rrsb, Stochastic-sb On.(fiss)a 
or sir u oj sdlsmrr surra ^ssu—Gujdq] sfiKfnd. sjGssrssflso Batch-so mini batch-sir srsmsssfldsma 
Giurrf^ srsmsssfldssiadro, oiuiorra ^ssuosu^rrsb o^(f)aoa srsirugi Ofogp Grspidi iSliq_d(3jLb. 
Giusoud Stochastic-sir susv>rrui_f&)sb arrsmuuQfii) usbGsurgj sjrofo fffrndardash sysuGsurrro, 
G^mo,§)]as$)SfTU uuj siru(£)£§)) ^jrofhid] suqTjsustnp ^smrjf^jdlro^i. 
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11.5 Hyper-parameter Tuning 

rflu^rrsb QrBLlQsurrijSleb <sp(ri)d)<so ^msiKj^dsish pmssmai &><siTG$)i6$)i_uj LD^luiShssxssrd 
ajDfgjdQarTshdUssrjDissT. epqjjlidso ^sns^Q^dai^d^, ldUbIulSIgsxsst rsmb <su<ss)tjujrr)]d& 
Qsusmu^iLjshsn^. <or(i)i£§]d&inLLi—rTSi weights, bias GumssTfD ^<srr<5ij(rijda(!snjd(3j, gisijd&gSilsb 
s^Q^dlsc ld^J u li suss)err QrbiLQsunij&li—ih oSli^QistflQSilGinrTLD. iSlssrssnj ^i&svsu 

uuSlpdluSlsirGurT^] pmssm <s &iflujmssT LD^luLianssxsnu QujnrgjdQamsrrdlGSTrnssT. ^ssmsb «^0 
QrBilQsurTijSleb stsusi/otsi/ <°>i®d(aja<sh £g)0<s.s CSs ii<sm@Lt), ^surorSlsb <oT^ss)ssrnodes ^)0«« 
G<5u<sm®LC> (oTssruem^GhuecscrTLCi pnihgnssi <su<ss)tjuj(r)]d&i Gsuem(^Lb. ^shsurTrorrs, pmssmsid 
KfOrryd Q&msrrGrmLDsb, pmc* ll^IuiSIssvsst suss)piuQjd(^Lb msi/ (j^dsi^dt^ hyper-parameters 

gtgstpq} Quiurf. Layers-esr <oT<sm<sstifld<5S)a, Nodes-esr <srsmsssfldss)-9i, Mini batches-esr <oT<sm<s$tifld<s$)a, 
Gradient descent-sc uiu<ssTuQ)d,^,ijuQ)ii) s.fnrnsoids.rTQsr oSl Sl^Lb(learning rate) GurTesrjD<sufDes)JD 
^l^jDarnssT loTQggjdairTLli—rTad Q^tT&xsosotriL. £§) <sujd<ss)JD fbrrd) ^fleup ^smmd^LCiisuGinp 
uebGsu^j LL&tluLi&asxsn spsijQ(surTssrfDrT<3id G&rT^gsjiu urrijpgj ^fliumssnss)^ G^psif 
Q&iusiJGnpGuj hyper-parameter tuning snsirdlGfomi). £g)g/ usbGsu^j (Lp6G)rD&<sfi\<sb 
r5<ss)i—Qu^jd}fD§]. < 3 i<ssxsu iSlssrsaq^LDrrrQj. 


11.5.1 Grid Search 
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0 0 0 0 0 
o o o o o 
o o o o o 


6£0 ^sssfluSlsb uebQsu^j ^snsgq^da&sxsn ^ 3 I<suidii 9 <s 61 (! 3 [B§] spsuQsurrq^ combination- 

G&rr^gSfiu urrijpgi <sp<ssr<ss)jD O^/jsiy Q&iususm^Guj Grid search GTsirdlGtnrTLb. 

GprjrsQ&Qdaiuui— Gsusmn^m ^ snsg (r^ da Gift sir Gremessfldes)& (^suro surra ^(r^d^idGurr^i 
$)LCi(Lptss)fD dlpuurra ^suLDiLjLb. 


11.5.2 Random Search 


0 

0 

00 

o o 


0 


0 

0 

0 


0 


6775<5 6^0 GU <SS) IJUJ <SS)J1) ILj L£) SS)SU^g]d QarTSfTSnmDGV, SfJfDfD (Lp6S)fDuSl(SC ^SMSljqTjdaiSSXSrTg 
G{ 5 rjpQt 5 ®d(&jLh (LpanrodtO) Random Search srssrrQ] Quiur}. drrrofD (LfiGS)jDuSl<sb 
GpijrBQ&Qdau uLli_rT6£j ld, ^jgiGisu gridsearch-gg <si5)/_ dlfouurra ^GS)UULjLb. 
GprjrsQ&Qdaiuui— Gsusmu^iu ^sfTs^q^das/rlsir srsmsmfldstua ^SdaLnrra ^(nyd^id 
uLL&d>§)l6b £g) Lb(Lpes)fDGS)UJ a,mb uuJssruQ)a,a,sc>mb. 


11.5.3 Linear Scale 


0.0001 0.1 0.4 0.8 0 9 


Ggouj uaGifl gst (oTsssrsssflassya, Nodes- gst GTGSsrsssflaGsya Glj rrssrjr) (8jGS)jr)Gu itsst GTSssTsssflasmauSlGO 
c 3 i<ss)i£>iLjih < 3 iGfTSH(n)da>GiS) 6 fTd > Gprfsg Qaiuiu ^jLb(Lpss)/Dss)UJ a,mb uiuesru^^Gomb. £g)<$6S7 
ldS ilULjasrr non uniform- ^a ^itsmuuLjLb. [Eg: 0.001, 0.1, .... 0.9] 
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11.5.4 


Log Scale 


0.01 


0.1 


1 


0.0001 


0.001 


&JD/D<s£ldamssr sfil dip ld (learning rate) GumssTfD < 3 i§>l(ydduj LD^IULjassrmp Gprj^l Q&iuuj 
^jih(Lpes)pes)UJU uiusiruf^^scrTLD. sjQesr&sfUsb ^piGumsirtn Ld^luLiaeiflsb spcp @)n$hu 
^erroSlsc lditjdjdld sjrouLh^rrsv Qumpp QpLdQsunijdssr Qaiusb^lpssxssriLiLd 

urr^lppiaSlQiLb. ^aGsu £ g ) Lb(Lpiss)jDuSlsc LLSduLj&Ghuniform-^a 0.0001, 0.001, 0.01, 0.1 ioTgstu pj 
Qumssr^j ^ssnirsuprrsb, ^j^lsSl(pppi spsuQsurrssrrDrrad Garr^lppju urrrjppj Gprfsy 

Q&lUlUSOrTLL. 


11.6 Optimization Techniques 

n5LDp] GlpLLQsurTrjdesr Qaiueb^lpesxssr ^i^lafluuprD^rTa dlppp hyperparameters-gop Gprjsii 
Q&iuiuld Gprrddsc usbGsurgj optimization suL^lsus^aaeh uiusirudppuuddsirpssr. Stochastic 
GD yxsoid sSls&rj surra &».®GS)a <3i67su_uj (yr^darrsonh. ^ypj pssrdarresr umss)pu51sb usoGsuryj 
gjjdjd ^pdarhiassxsnd QarTsssri^rpuus^pd arrsmscmd. 



^ipssxssr Qaujpj drjrrssr (Lpss)fouSlsc &n.@<ss)a <3i®r)i_sua,!D(3) ^psu iSlssrsurpid optimization 
surfsussraash £g)/5V0 uuJssruQippuuQidesrrossr. ^jpesr ysoid dlppp ^srr si^pdash 
G>prjr5Qp(i))dauu®d6srfD(Ssr. 
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Gradient Descent with Momentum loTsiru^i ^g)/^)( 3 si 7 «/_D/T« &jjrT&ifl<SG)iud (exponential 

moving average) &6mdd) lL® u u uj sir subsist ^tpeoih (^i^sm&UJ oscillations-®od 
&nl($)uu(i)it5t5 (LpiLKsbdlingi. 

Root Mean Square propagation GresruGg RMS Prop (orsirfnismLpdauuQdlingi. ^i^rrsu^] Qpso 
dFjjrr&ifl &3)HJ LD§)uiSlG$)65rd &6 sst(J)u 3 <s>l£<ssr ^Lpsoih urjsfi), oscillations- god 
<3nl(^)uu(^^^l dljors^ <sq6fT<sq(i]jda6S)<sn£ Gg/jrsQ&QuuGg RMS Prop 
<oTGST[D6S)LpdaUU(i))d}fD&i]. 

Qll(d&*-pSIiu ^jpem(S) p£&q<5iJ[h]a<st?l<siT acsouGu Adam optimization ^ 0 /A 


Learning Rate Decay 
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Constant Learning Rate 




6£0 Q^lLQgutttjQgst cost ld^Iuli gradient descent QpsoLD a^f^GSiai < 31 <ssm_uj (LpiusyihQurrSfi, 
<313 ,gst G)um£g aryinridlaGiflGOiLCi ajofosgidamssr aSl&tlgih gtgstu^i LDrrfQrra, spq^ ld§>] uurr& 
<3jGS)LD&}fD§J. GTGStGgU <31^ GST 3rLpfr)dl3Gff]Gb GyGSTPSlGST GuTT^l COSt LL^IULj &*.($)GS>&d(&j <31(T^dlGV 
^GS)LDrB^rTGO 3&.L-, <31^<SST 3fnp<SVld33GST LDTTIT)TT3) LD^IUGSlUU QUITrQJgSfl 33)311 rfjGSST® <31®^ 
<3]LjyGSnU GT®^§1 GSlGud&IDgl. GTGStGgU 3^®GS)3i 3LDUJp®GV <3jgGS)GST G)S1 lL® GrSlscSld Q&GSTfQJ 
<31®d>9>®d>{5 &LpfD®&6lfl<sb L£>lL®GlD a^®GS>aGS)UJ ^IGS)l_ld)fD§J. ^JGlJGUrT^J a^®GS)3i &LMLj£GilGb 
G^gs)guuS1gcgoitlcgc r£}aLf>£guu@Lb ^tglauuLq-UJrTGsr &(£fD!d)&GS)Gn£ ^GfilijuugiDarTa Gurs&Gg 
learning rate decay ^,0/A 3rn3)<svid33GST sfildlp ld®ugs)U <gasi/Q<si//T0 aTLprndlu9lGpiLb 

Q&tt^&ld QarTtGTj&LDrra; (^GSip^^jd Q&rrGssrGi _ guq^ld. gtgstGgu a^®GS)a; 3 lduj^®gst Gurrg] 

3, GST3)1 <3]®$$®$$ <31Uj_3GS)GfT dlGSTGST{GT) dlfShU^33 GT®^§1 GS)GU U UgTTGO 3TGCU^^lGC 
3s- L ®GS)3GS)UJ <31GS) 1^3,3)1^0)13. ^]pGST3GV G^GS)GUuS1gCGC 3LDGC [£l&(LpLf) <31®3U U LpUJ PTGST 

&rbrr)i&iGh £6fi}[jda;uu®§ilGSTinGST. 


11.6.2 Pruning 

^3)i Q^lLQgutttjSigst Q31ugc^}idgs>gst <31$5l&lflda 2_<$6iyi£> «£0 GULflGUGS>3 ^(ajih. gt®£pgiii_gst 

Qt^iLQgUTTTJ&S)^ <31® 3 <31GYTG11 [§)gl ) rrrrGST&(Gnjl—GST GUL^GUGSHDUU^aGC <313,GST Q&UJGbSilfDGST 

Q^m^dai t£1gs>gouSIGgoGuj (Lpi^d3uu®®p^i. Pruning gtgstu^i Q33(g^3lc> Q33(g^3ld33 
QrBLlQGurrrrdlGV ^_gttgit rffu^rrrrGST&GtflGST GiGssrGssfld gs)3gs)uj <3i®3ifld3 s ^0 gul£1gugs)& 

^,0/i). ^j3>GST ^LfiGCld QrBLlQGUTTlfdlGST Q3LUGV®l3)GST LI 3® 33 U LI l _ rTLDGO umj^3)ld 

Q&rTGTTGfTGVTTLb. 
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11.6.3 Batch Normalization 

Normalization ufjrfl gtgitIuj ^lSIl^Igo gtgstjd 'Multiple linear regression' GTGSiiif u( 3 j§i)uSlso 

aSlGtrddufilqijuGuGST. < 3 t&rrGiiy <sp(r^ Gfiifispasr GiSlGsiGOiurrarry, <3]^gst &yp < 3 iig Gi^Gapif wroyif 
< 31 Gurnfil go i Giron < 31 onmsiGiflGrr GTGSsrGssfldGnai Groriiif 2 input features-gou Qurrypy 
<31 ssi ld d id Q^Gsfl go &y p <31 Lguj nasiy 450 -sqft (Lppsb 2500 -sqft gugsip urjaS)uS](njd(&jih. ^gstitgo 
< 3 lGS)fD& 6 ifl 6 ST <oT<sm<s 5 tf\d<ss)&GujrT 1 (Lppsb ^(f&uif&if 5 <sastrip ldlLQGld upaShfil(rijd( 3 )Li). 

£g)<si/ Gurry GlGUGiiGGiiy GTGOGriGO&GiflGb upGiSluflp^d^if input features- gi^gsu^uj ^pGp&Gr>Gn dprra - 
1 (Lp&)GO +1 GUGnp <S 4 Grands, (Lplu goguG^ Normalization GTGsrg y &GmGumf. 

srr^rrpGsrrregression-GO QsuJGOu(fiif Normalization-go u GuttgoGgii rffg^pab QgifQGurrpdGO 
QsuJGbu(f)Giiy 'Batch Normalization' ^(^if. GpQGSTGsflGO QrpLLGlGimptajdt&j uufrrjd ^Gtfldsd 
QsGVldsuudif ^pGijSGfr mini-batch gradient descent QpGOif QsGpidsuudif Gurry ^gsigu 
UG oGGiiy Q^rr^^lsGiflGO ^GsnndGsrgiGsr. gtgsiGgij GpGiiQGurrqrj Qsmsd uf go i if ^GnunLjih 
^pGpsGir UGoGGuy gtgogsigo GUjflGsissGiflGb ^(njuugjDarTGrr GamuuLjSGrr <3i^3Jif. ^]^gst 
G iSlGriGfTGurrs, <5jrr)u(f\GijG^ 'co-variance shift' (Change in data distribution of the input features) gtG rind 
iSlpSSGSIGST <3^(^lf. GpQGTTGsflGO G£GilQGUrTQT) (LpG5lg)Ug if rflUgprTGrTSGIT QgUG il Gaily ^pGp 
GTGVGr>GV<9i(®Tjd(& ) GJfDU S,lf(LpGS)UUJ <3jGnGp([ljdsGr>Gn LDIJ/Dlfl <3]Gr>LC)ds (LpiUGOuf. if (gp GS)g) llSl GO 

uuflrDyGdidsuu(f)Giiy sfhurrs (gf\prry. gtgsiGgij 6£<si/(o)<a//T0 QsmsdsGifiGOi if s^GTTGn ^pgi^sgtt 
<31 (a6UUJ0«0 <3iGiniuuuu(K)Gu&rnc§ (jpGsiGsrp <3t®r>Gu normalize Q&iuujuulLQ}) 

<3lGS)iuuuu(f)GiiGs 'Batch Normalization' GTGrrry <3]Gny)dsu u(f)dfoy. flu^pGO GlrBifQGurrpdGb 
^jGsiGiJ QgrT(3j§) Giinfliurrs [BGriuQuryGiJ^rTGV (gfiy £g )uQuiurflGO ^GriLpdsuuQidrny. CNN 
GurrGSTjD Qr5L-(olGijrTp(ajd(3j uuSlrpd ^Gifld^ if Gurry srr^uLj-QGUGirGnGnij uurhisGsiGtrd 
Qsrr(f)fy uufrodl <3tGifluuyif, iSIgstgstp gugsstgsstu LjGnsuuurdsGriGnd Qsrr(f)fy sGssfldsd 
Q&rrGVGViGiiyif (sfjdt^Gn&uj 'covariance shift' iSlpd&GS)Gsrd(&j S-^rrpGssnhisGirrrs <31Gsamigif. 
GrQGsrGsflGO ^Gu/D/flGrr iSldsGO LD^iuLisGiflGb <3]§)s <3]GfTGii GGiiyurr® srrGmuu(fuf. (CNN 
urjrjrjSl ^GsflGuqrjif u^^lsGiflGO GiSlGnssLorrsd srrGssrGomf) 


11.7 Example Program 

deep neural network- go rsnif uuJGrru(f)f^iLjGhGfT umpusu iproyGgrriLidsrTGrr 
Gr(f)£ydarTLLGSii—Guj ^rd^if uuJGrTuQ)s,^nigGhGGnrrif. difdsGmu ^jurhisGifiGO ual(fiif 
flpab srrjy GifftdlLurr&uuddfDy. 


1. umSilflg ^pgi/sgit train test split gpGOif iSlfidsuuLfu 9 _i_gst, X_train-GO 2-GrrGfr (Lp^GO 2 
features LDilQif SGSsrddrrgs, Gr(ffyd QsrTGfTGnuu(fidrDy. lSIgstgstp ^GiJGiflpGssr® gpGps(Gnjif 
non-linear gtguG urry (LpasipuSlGO <31 sn ld rsy GrrarTGrr GTGsruy GUGnpuuif ^pooif GUGSiprsy 
3jmfuuuLL(fiGfTGny. 
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2. hidden Qsciuflsc s^shsrr spsijQsurrq^ node-id (Lprsempiu Gsviutflsvimm node-sSl^fgj suq^id 
ld§>] u Li i^sst weight-gou Quq^dSl ^siijd^JI—SST bias-god 3hi.Ll(j})id rfl&Lfxsij tf.matmul(), 

tf.add() ^LpscLD rBStni—Qunpj&tngi. ^fshsurrrrp &smddli_uuidL_ ld^JuQu hidden Gsoiufflsb relu 
Qpsoid activate Q&iuiuuuQSilfDSfi. as$)i_dl Gsoiufflsb ldlL@lc> activate Q&iuiuuui—rTg 
^jiliLD^IULi, cost &<sssr(i)iiSliq-&&& Q&sgigpuuQdlfDg]. sjQssrssflsb cost-sir s ^0 u(&,§)]iurra 
3>sm_&} GsoiuQTjdarTssr sigmoid activation r^ss)L^QurDQ] sdUfidlro^] srstrigj sprDQ&ssrGsu 
umjfG^rrid. ^j^ss)3iUJ forward propagation (Lpss)piunssr§] spp^ gssf\ function- 
<oT(Lgg u u l. (/j) GrrsrTSfj. 

$l$ 5 lsb dropout optimization srsiru§] LDS$)fD(Lp& ^(f)d(^sisiflsb, relu activation r5SS)i_QuiD/D i/drD(3j 
spd^dJso $u^!jrrsiT&s$)GfT, QarrQd&uuLdi— (Lpi—da stfldl££$rn(&) srtDtDfTijGussb Q&iusflLpdand 
Gl&iudlssrjD (Lpss)jD ^(Pjid. £g)<$/D<s/T<ssr rflrrsv lSIsstsijq^lcis^j ^]ss)smdsiuuLd(fehm^]. 

if drop out == True: 

A = tf.nn.dropout(x = A, keep_prob = 0.8) 


https://sist.sithub.com/nithvadurai87/4baef96f858284dla4868c80df25f990 


import numpy as np 
import tensorflow as tf 

from sklearn.datasets import load_breast_cancer 
from sklearn.model_selection import train_test_split 
import pandas as pd 

def normalize(data): 

col_max = np.max(data, axis = 0) 
col_min = np.min(data, axis = 0) 

return np.divide(data - col_min, col_max - col_min) 

(X_cancer, y_cancer) = load_breast_cancer(return_X_y = True) 

X_train, X_test, Y_train, Y_test = train_test_split(X_cancer, y_cancer, 
random_state = 25 ) 

X_train = X_train[:,: 2 ] 
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import matplotlib.pyplot as pit 
import matplotlib.colors 
colors=['blue','green'] 

cmap = matplotlib.colors.ListedColormap(colors) 
pit.figure() 

pit.title('Non-linearly separable classes') 

pit.scatter(X_train[:,0], X_train[:,1], c=Y_train, marker= 'o', 
s=50,cmap=cmap,alpha =0.5 ) 
pit.show() 

def forwardProp(X, parameters, drop_out = False): 

A = X 

L = len(parameters)//2 
for 1 in range(l,L): 

A_prev = A 

A = tf.nn.relu(tf.add(tf.matmul(parameters['W' + str(l)], A_prev), 
parameters['b' + str(l)])) 
if drop_out == True: 

A = tf.nn.dropout(x = A, keep_prob = 0.8) 

A = tf.add(tf.matmul(parameters['W' + str(L)], A), parameters['b' + str(L)]) 
return A 

def deep_net(regularization = False, lambd = 0, drop_out = False, optimizer = 
False): 

tf.reset_default_graph() 
layer_dims = [2,25,25,1] 

X = tf.placeholder(dtype = tf.float64, shape = ([layer_dims[0],None])) 

Y = tf.placeholder(dtype = tf.float64, shape = ([l,None])) 

tf.set_random_seed(l) 
parameters = {} 

for i in range(l,len(layer_dims)): 

parameters['W' + str(i)] = tf.get_variable("W"+ str(i), 
shape=[layer_dims[i], layer_dims[i-1]], 

initializer=tf.contrib.layers.xavier_initializer(), dtype=tf.float64) 
parameters['b' + str(i)] = tf.get_variable("b"+ str(i), 
initializer=tf.zeros([layer_dims[i],1],dtype=tf.float64)) 

Z_final = forwardProp(X, parameters, drop_out) 

cost = tf.nn.sigmoid_cross_entropy_with_logits(logits=Z_final,labels=Y) 
if optimizer == "momentum": 

train_net = tf.train.MomentumOptimizer(0.01, 
momentum=0.9).minimize(cost) 

elif optimizer == "rmsProp": 

train_net = tf.train.RMSPropOptimizer(0.01, decay=0.999, epsilon=le- 
10).minimize(cost) 

. . . . 
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train_net = tf.train.AdamOptimizer(0.01, betal = 0.9, beta2 = 

0.999).minimize(cost) 
if regularization: 
reg_term = 0 
L = len(parameters)//2 
for 1 in range(l,L+l): 

reg_term += tf.nn.l2_loss(parameters['W'+ str(l)]) 
cost = cost + (lambd/2) * reg_term 
cost = tf.reduce_mean(cost) 

train_net = tf.train.GradientDescentOptimizer(0.01).minimize(cost) 
init = tf.global_variables_initializer() 
costs = [] 

with tf.Session() as sess: 
sess.run(init) 
for i in range(10000): 

c = sess.run([train_net, cost], feed_dict={X: normalize(X_train).T, 
Y: Y_train.reshape(l, len(Y_train))}) 
if i % 100 == 0: 

costs.append(c) 
if i % 1000 == 0: 
print(c) 

pit.ylim(min(costs)+0.1 ,max(costs), 4, 0.01) 

pit.xlabel("epoches per 100") 

pit.ylabel("cost") 

pit.plot(costs) 

pit.show() 

params = sess.run(parameters) 
return params 

def predict(X, parameters): 
with tf.Session() as sess: 

Z = forwardProp(X, parameters, drop_out= False) 

A = sess.run(tf.round(tf.sigmoid(Z))) 
return A 

def plot_decision_boundaryl( X, y, model): 
pit.elf() 

x_min, x_max = X[0, :].min() - 1, X[0, :].max() + 1 
y_min, y_max = X[l, :].min() - 1, X[l, :].max() + 1 
colors=['blue','green'] 

cmap = matplotlib.colors.ListedColormap(colors) 
h = 0.01 

xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h)) 
A = model(np.c_[xx.ravel(), yy.ravel()]) 

A = A.reshape(xx.shape) 

pit.contourf(xx, yy, A, cmap="spring") 

plt.ylabel('x2') 
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pit.scatter(X[0, :], X[l, :], c=y, s=8,cmap=cmap) 
pit.title("Decision Boundary for learning rate:") 
pit.show() 

p = deep_net(regularization = True, lambd = 0.02) 

plot_decision_boundaryl(normalize(X_train).T,Y_train,lambda x: predict(x.T,p)) 

p = deep_net(drop_out = True) 
p = deep_net(optimizer="momentum") 
p = deep_net(optimizer="rmsProp") 
p = deep_net(optimizer="adam") 


Non-linearly separable classes 



0.8640964968487806 

0.6576789363554733 

0.5143526817019538 
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0.46109938219842395 


0.4391590019944309 

0.4253497158288082 

0.41486545995180873 

0.4064618870927915 

0.39961302157076345 

0.3940068025490444 
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Decision Boundary for learning rate: 



0.7001094555968302 

0.49777905860234667 

0.36297676798298484 

0.3099069894603214 

0.2926005495411737 

0.30422081040917726 

0.279327552972575 

0.2927437389848652 
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12 Convolutional Neural Networks 


6£0 database-<50 G&L&uupfDtgj 67JDJD 6U etna id) 60 ^rrsii&ansnd Gl&rre sarz_ ^6S)LDUiSliD (g 'Structured 
data' 6T6$Tir}j Quiurj. (ff&tfsijstnrr rsml urrijpp ^6S)6si£&jiih sgqtj (LpenjDiurrssT 6UL^6ii6su£>u6S)ud 
Q&rrsmi— ^rrsii&stnm Qrsi-LGleurrrj(3jd(&j Qaintfi^gi gt si] so it try uu5)jd@ ^sifluu^j stgst^j 
umjfG^mh. £g)<s3f?6iv0/i> ut&jtfl&sftleb <sp(Lgrii&/D/D ^ysifs.^ds.rrssT LDmsb&smsn srsdsurTf^j 
^_ 0 siy/T« 0 si/^/ 6T6&TIQJ urrrfdasoml). 

<^lL-i 6uss)6ssr su uysu d>&il so 6$)LDu_jid <$/ 76 iy<s 6 Yr 'structured data’ sresiforreb, ^)si/si/^_si5)su G&ifldai 

^lueorrg ui_td&m, arrGlemrTeifl&err, 2_s$)rj ^rrsii&sh, (ajrjec Gs.rruLjs.sh (images, videos, text data, 
voice data) ^dliuonso 'unstructureddata’ ^(^ih. CNN, RNN Gurrssrrossxsii ^^esianu 
Gsustnso&isfjjd&rr&u uujsiru(f)dlsirrDssr. (ffistnsu&sinsfTU u/b/flQiusvsomi) ^ssflGUQTjLC) 
utsjSiJaeifleb urrjjdaeorTLD. 

<^$5)a 5 u i [hifEQSKofrd Glarrem® uufljndl Quid try l /^)^/ t « si / 0 ® s 57 vd s ^0 "stsstsst ui_Lb? ” 

6T6ST 3>6ti&fld(8jLC) G6U6S1606SWJ CNN d)pUUIT3i Q&lLldllDgl. QUITSfl6U /735 Ul_[h]&>6$)SfT 

6U6$)3iuu®£§)l ^fssH _ iLUT6fTL£) (Image classification & identification) s.rr smi sus.farrssT S £0 %l)l; 

6iks$)3> sol^sossiiduiSIgo (ff CNN S-qtjSurrd&uuifQshmg]. ^jsu/b/flsb uuj6siu(fi^uu(f\Lb 
%ul/ susm&u u&jihJ3isir fT6ST Convolution, Pooling, Striding, Padding, Flattened array, FC (fully 
connected network) Gunssrpesisu stsstQsstsstsst eustnai&isifleb Q&UJ<svid(&, a_^siy®s577Dssr. ? 

^siifD&ofoQuJGVsorrLl) ^i—rhi&eifleb ums$Tu(ff^ G><suernfftib? GufTssrroasvsii 

ujoj^Ghuebeomb ^juu^^luSleb arresBreomb. ^j^ssr 3h1i_6S)ldul] iSlssrsuqijLbnrr)] £g) 0 <s 0 /A 
^j^leb s^shsrr spsbQsiimi^ uis^ssiiuiLjih aSlsnd&uin&d dG>Lp ansmsomb. 


103 




Padding 


ULSf.1 - Input Matrix: 

(Lp^eSlsc <sp(rt) ui—£<ss>p aessflssfl L/flrs^] Q&msfTGU^fB(& ) stjdjd Ginstn&uSlsb srsusurr/rp 3 i<ss)i£>d& 

G<51J<Sm(f)LC> (oTGSTUgl IT,LD&(&)&, Gl^fl[5&il([ljda G>GU6m®LC). &GSsfl<Ssfl(SS)UJU Ql//T0 ^ GUSSlfJ tSp(ITj 

z_/z_zi> 67WL0/ usbGsu^j iSld&tsv&snmsv ^(r^sumssr^]. gjfirB&u usbGsu^j rgljnu iSld&sbsussKsrrd 

Q&msssri— ui _ d> 6 $) 3 ) ueoGsu^j ^rsmanssxsnd Qamsssri— g ?0 3 i<ssnflimra urn/D/fl iShssnssrijgmssT CNN- 

<350 2 -filUJ UScQgU^J G<SU<SSXSO&<SSXS{t£ §l<SUrhlSi G<SU<SSST(f)Lt>. RGB <oT(S5TU§l (Lp$<S5T6$)LC> 
susmsmrd&m. £|)esxsu <spdjQ<surr<ssTfQ]Lb &hsst<ss)i<ss>L-UJ shades-ssi ^aS\p^<ss)^u Qurrrrpd.&rf 0-255 
GUGtnp ^<smr>r5& <5Tsm3isnrr<sv (&jn$ld&uu(f)Li>. ^jgiGsiJ colour codes <oT<ssTp<ss>Lpd3iuu(fdlp§]. 
^jd> 3 ) 6 $) 3 iuj <oTG$sraiGi?l<ssT asouGu usbGsu^j r£ljnu LSld&sb&Grmsb 3 t,<ssT ui_£<ss>£ s ^0 3 ^<smfliunsi 

umjDjo 2_<$6iym rflu^pso QrBLfQsurrrjssisiu Qurr^j^suemp {g)£g<ss>aiu s^suQsunq^ iSld&sb 

LD^UL/LI) '<oT<SST<SST UI—LD?' <oTGSrUGS)^d & 6 Ssfld& 2 _^GpL£)features -^<95 31<SS)LOlLjLb. < 0 T<SS 1 G<SU 
£flf^<ss>&uj features-<ssT <oT<sssr<sssfld<ss>si<ss>ujd (&j<ss)(Dt 5 §i QrBLfQsurrpt^d^ 3 tssnuu CNN uscGsu^j 
<suL^\<siJ<ss)3i3i<ss)<snd <ss>&ujmsn5>lfD3}]. ^tssxsuanssxsnu urorfl fjfjuu (&)&>! uf<sv guTsssrsomt. (Lpp<s 6 )<sb 
< 31 <ss)UULjih 3isssfl Input Matrix <oT 6 STiD<ss)Lpdauu(f)!d}fD§]. £ 3 )< 5/00 31 (fd,^, ULpiun& 'Convolution' 
<oT<ssTfD <sp<ssT(T)] it><ss) i_Q u fQ(Qi features- <sst fSTsmsssftdssi&issnud (^tssijoddlfD^i. 


ULp2 - Convolution: 
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^jLIUL^llSUsb r5LC>(Lp6S)l—UJ U L _ g§)l6$T U ift LD IT StSSTLC) (g<S$)pd&UU(j)ldpgI. ' U fi LDfTSmgStDgd 

(gGS)pgg<sb' loTSSTJOITGC UI_g6S)g (STSlJsfigddsVI LD Q(SUlLl< 5T(fi)damf)<5V Qsvpjlli QufillU ^gJ/OTSiy 
ui_gG$)g drfihu ^jerrsfilsb umppjGiJstngd (gffildtgih. ^\gssr QpsoLb ui—g&ylsir 
fgesypdgrrspiLb. ul^(D@ (oTsuefilp ^LpuLjiJj erpui—rrg]. ^jg] ifilsstsutgih (LpsnjBifilsb 

rflaLpdpg], 


(LpgsShsb 2*2 ^sbsogj 3*3 Gumssrp dljfihu suLq_suss)LDUss)ud Glamsmi— spcg ^sssfl 
suistnrjujpjd&uuQdpg]. ^jgjGsu Qufiliu ui—gS^sir uflLorrsmgstDgd (gempda s^gsi jld <sp(g 
(SUU\.3nLis^(filters) / ^<sn<s^p]dai<sh(parameters) ^jg] filugrrsv QgLLQsarrrjdsb it, mb 

uuJssru(figg]Lb weights-dgd &ldld rrssrgi. 


Scharrfilter, Sobelfilter Gumssrpsupemp g^ggetnaiu Gsuosxsv&^d&rr&u 
u uj ssru (filgg so mb. Scharr filter mssrug] dip] dip] ug^lsiermsiu iSlfidauuffiLb ui—gtzhsir 
spshQsurrg ug&>l (Lpesxssr^^gdgLb (LpddiuggisuLb Qarrtdgg] (suigaLbig rnffidgih. 
^jg/Gsu 'Edge Detection' (oreirp] ^]&S)Lpd<3iuulg)dpg]. 

Sobel filter mssrug] spg ui_gd>lGST g&lduju ucgduSlwimm iSld&sb&nggdg ^]$5l& 
Qpddiugg]<suLb Qarriglggi ^gssrL^uuemi—uSlsb si//jj_<sL(J)/i>. g^ggetnaiu suLps.ifiiguSlsbr 

LotglULj&SfT lSl<S5T<SU(njl£>np]. 

(oLDGVILb ^jgg 6ULp&Lb(filLb fil&Lf)<5UmSSTg] dloS)l _ LDlLi _ 6U irddsb fil&Lp Qsuem(fi)LCirT 

<3l<5V<5Vg] Gl^ihjgggisurrddsb rfi&Lp GsusmtglLbrr mssrusmgu Qurrpjgg] ^jsnpjfilsb 

rSd < oS)L£>U-fli> SriflUJlh LD^IUL/Sim L _ LBLDITp]d<SSTpSST. 

^IGSxsuasrrtsbtsorTLCKsb grrGLD <*«./_ rsLodGappisii rrpj LDtfiluLj&esxsnd Q&rrsmi— spg 

<suig3iLbig<ss)UJ ^(gsurrddu uiu<ssiu(fi\gg<scmb. ^jg/Gsu gLb(jp&s)i_iu lditi _ <sv 

uiusiru(fi)gg]deirp parameters <oT<s$tGgij back propagation (Lps^puuig ^jg] 

gesrdamssT &fhumssT LDjgluifihssxssrd &pp]d Qarr<sm(fi)<5fil(jb)Lb. 
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Fi Iters 



Scharr 
3 0-3 
100-10 
3 0-3 


Sobel 
1 0-1 
2 0-2 
1 0-1 



-101 
-2 0 2 
-101 


-1 - 2-1 
0 0 0 
1 2 1 
Vertical 


Horizontal 


^IglGumssrfD GUL^anlL^uSlisir ^snsflec Qunqjjtpgiih ul^ fBii)(Lpss)L^iu 2_<sh<sr?Ll(S) 
^sssfliumssraj] (Lp(L£>su^iih dlfQj &\(Q] u(&j§il&<snrTaiu iSlfds.uu^dlfogj. 


dipd&tsmi— <oT®£§idarTLlLii-<5b 3,5,2,1.... <oT<sst£ ^jsuihJ^LCi ashsifLl® ^sssfliL/Li, 1,0,1 <oTss)ilL 
ld^uiSIqdsstu QurQfQjd, gji<surhi(3jii> 3*3 filter ^sssfliL/LC) sues)ijuj^jd3iuuLl(f)<sh<sn^i. 
&><sirG$)iG$)i_uj ^errsflsc Qurrq^^^iih uis\-um&i <srrGi?LL(f) ^sssfluShsir ^i—g] QpssKSVifilsb 

Q^m_fhl§ll Ul _ LD (Lp(L£GlJG$)£lLILD dlfQJ &)tgj U^&h&mrT&U iSl ffldsiUU (f\S)fD§l- (Lp^<s61<SC 

Lii _ d)§)l<o&T _ §1 QP<ss)60u5)6fi([ijr5&ii 3,5,2 OTgo/zi) ld^IulSIgsxsstu Qu/orruf ^jsuihi^LCi 3*3 

^GSsfl<SS)UJ ioT(f)dd}fD§]. 
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Convolution = [(9-3)/1] h 
(7*7) 


^4|2223]Pr429l24 
I p4 1fil 30 
23 11 1912 22 23 26. 
22 12 2313 25 25 29 
14 22 21 22 23 22 33 

25 13 29 24 30 26 22 

26 301818 2919 23 


Pooling by (2*2) 





10 ^3 29 
23 23 25 
25 29 30 


14 21 23 33 Avi 
26 18 29 23 

Convolution = [(9-3) / 2] h 


24 22 
20 27 


Pooling by (2*2) 


iSlssrssnj ^sssfl LDfo^jih Guis^aaLis^ ^sssfl <3 tss)Ild ^GuaSirressT® ^sssflansiflesr QuQTjdaeb 

r5<ss)i—ClufD^j l]§ i)uj ^i<sssf}es)UJ li^Iuj ^jsssfluSlsc ldG$/_}/_/« err 

^gs)gst£§]lc) &^lLi—UulL® 30 <oTgs)ilL dlfShu srsm suls^g^Isc (SjjfildgiuuQdJjD&ii. ^gjGsiJ 
"Convolved output" <5T<sirrD<5ir> Lpd&u u ® ll ^sssfluSlsb ^ji^LbQuQjLD (Lppsb LL^Iuurr^Ln. 
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352 101 

3*1 5*0 2*1 

302 

Cal-1 

= 085 *01 0 = 

0*0 8*1 5*0 

= 0 8 0 = 3+2+8+9+8 = 30 

30 

918 101 

9*1 1*0 8*1 

908 


521 101 

5*1 2*0 1*1 

501 

Cal-2 

4 A 

=857*010 = 

8*0 5*1 7*0 

= 0 50= 5+1+5+1+2 = 14 

14 

182 101 

1*1 8*0 2*1 

1 02 

Cal-3 

212 101 

2*1 1*0 2*1 

202 

22 

=578*010 = 

5*0 7*1 8*0 

= 070 = 2+2+7+8+3 = 22 


823 101 

8*1 2*0 3*1 

803 


918 101 

9*1 1*0 8*1 

908 

Cal-4 

= 7 1 4 *010 = 

7*0 1*1 4*0 

= 010 = 9 + 8+1 +0+5 - 23 

23 

025 101 

0*1 2*0 5*1 

005 


^GijGurrtru "Convolved output"-sir ^®*5(5(®(5(5 ld^uiSIssisst S-Q^surrd&u Gurr^id ^sssfl (Lpgsb 
^GStifluSls&QTjrBgn srshsusnsif £g) ssn_Qsusrfl G)5 IlL(® G^prsQ^^dsi GsusssrQ)LC> srsirussi^Giu stride 
stssiild parameter (^(Sld®fD§l- ( oLDfD&sssri— sr®^^]dsimdLpsb stride (S=l) STSSiiihGurra,! (Lpgsb 
^GSsflifilsfiQjjrBgl ^)ss>/_Qsi/syf? s£Il1(® 5,2,1 stssiild LD®uss>ud Q&rrsmi— r3i®*5t5 ^sssfl 
GgrjrsQ&QdaiuuQdliDg]. ^i§iGsa stride=2 STSsiiidGurr^i (Lpgsb ^sssfluSlsSlq^rB^j £g)/r sssr® 
$l<ssu—Ghsu<si?l qSIlL® 2,1,2 stssiild wSiluiSlssissrd Glamsmi— ^i<mfl ^®d,&, ^sssfliurrag 
G^ijrBQ^®d<9iuu®suss)^d dirrsssrsonih. iSlsirssrrf GprjpQtzQd&uuLdL- (gfiggssisiUJ 
^sssfl&ignjisir Gluqt/dasb, &^lCi_<sv ^dliussisu j^ss)i_QufD^i "Convolvedoutput" -sir 

mi^UL/ash 9-(ij<5ijrT<dauu(®®GsrfD6ST. sir surr/orrai stride LD®uiSlss>ssru Qurrfgjgaj] 
convolution r5ssu—Qu^j®fD§]. s ^0 n*n ^i<sssfl<ss)ujf*f suupanLisp Qpsoid sinsp&LLQiLbQurr&ji 
a_ 0 si//T 0 ii> (&jss)jo[5& lj rfl ld rrsm^ssi^d QgnTsssri_ ^sssfliurrssr g/ [ (n-f)/s] + 1 stssiild 
sumuuumdt^ssr ul rp ^jssi ldll/ ld. GLDfD&sssri— srQifd&jid&irTLdispsb 9*9 ^sssflssnu 3*3 suup&idui 
qpsold GULpaLlQndGurrg] dtssii—d^LD convolved output-sir ld^ul/, 


• stride=l srssflsb 7*7 stsstsi^ld i.e [ (9-3) /1 ] + 1, 

• stride=2 srssflsb 4*4 stsstsijld i.e [ (9-3) 72] + 1 , Gldib&.ssstl- uup ^ssiLDrB®(r^uussi^d 

&n (ofiftTGV fTLD. 
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$l&fD(8j uLSf-iun<9i 'Pooling' <oT<ssrues)^u ujDjfld ^rressreomd. 


ULp 3 - Pooling: 

tS£(Tl) Ul _ ®®I 65 T U®LDfT 655 T® 65 )^d ( 3 j 65 )fDUU§] 'convolution' 6 T 65 TII) fT 60 , < 3 ]UUI_®®l<dr < 3 l<SfT®SX 5 ud 

( 3 )<ss)fDuu&i] 'Pooling' < 5 T 6 $tuu®l£>. ( 3 )GS)fDdauuLlL- uflumessrid Q&rrsmL- s^0 

U L _ ®®l 65 T lS^] LSstskr® LD S£0 Q&UJSOU®®®, ^ff, 6 S) 65 T dfQ] dfQ] U(&,$®l& 6 fnT&U 

LOlffld&ingi. iSlssnssTg spshQsurrq^ u^^luSleSlQ^rB^iLD epqjjdJeo iSld& 60 & 65 ) 6 fT ldlL®ild 
G>^rrr5Q^®uu^<sir Qpeoih <3iuui_®®l<sir <3]m<sii (^65)fod<3iuu®dlfD§]. ^iiltqpGtniDifilsb 
LSid&tsbaissxsrTg GgijrBQjsQida uebGeu^j suLflsiJistnaaissxsrTPooling sm&ujmshdlfBg]. s^0 
<su (tpsoLC) ui—umtssraj] usbQsu^j u^^l&mrr&u LSlfldauuih— iSlssrssrrj, spsijQsurrq^ 

u(^®u 5 I& 2 Jld 2_<srrisrT iSld&sb&eiflsb Quffhustnp (3<s/7siy Q&ujsuqi, dlnShuemg G^psy Q&iusijgi, 
^svnsiJ&GiflsiT &ijrT&!fl LD^luiShssxssr^ <3<$/76iy Q&iusiJgj (Lps^foGiu Max pooling, Min pooling, Avg 
pooling loTssrigj < 3 i<ss)Lpdaiuu®)&) 6 STfD 6 ST. GLD(D#.<smt— <oT®)£aj]dairTLLLq-< 5 b 2*2 smp&ULp ^stssfl 
uujssTU®®^uui®®shm^]. <o7wG> si0 rrssr 2 *2 ^svlduiSIsv 'convolved output' -^sst^i dlfQi dlfQj 
L/0^)«syr/T«^ ( 5 ^/rsiy Q&iuujuuQ&tng]. 


(Lp^sSlsb 'convolved output'-sir £g)/_g/ ^LpesxsouSleSlQ^rB^] 30,14,17,26 <5765)1 ld Pild&eb LD^luLj&esxsnd 
Glarremi— dJ/ryu^® Ggrjr5G)p®)daiuu®)d)fD&ii. LSlesresrij ^j^leb 'MaxPooling' 
UlU65TU®®^UULl® 65)6U3i6lfl<sb QufllU LD^ILJUIT65T 30 6T65TU3jJ GgljrBQ&Qd&U U®)§ilfn§J. 
^isiisunigj <3l(®{5{5(®{5{5 u^^JuSleSlQ^rB^iid 'Maxpooling' QpeoLb LD^IULisnsrr 
Ggijr 56 )&®)daiuu®) 6 iJgrT 6 b ui—sir <31 sm ay (3)65)fDdauu®)d)fD§]. 


6® (R) lj u uup - Padding: 

^] 65 ) 3 ) aLh—rruJLDnda Geuessn^uj ^GudlujLSlsbesxso. firzi&sh sfilqTjuuuuLli—rTeb 
uuj6sru®®®ld QamsrrsrTsorTLC). Padding 6T65TU&J ui_®®165t rBrresr^ QPSsxso&siflsyLb StL^Iujld 
$65)fnr5& 61]$\65)&a65)6fT 2 -(nj 61 JrTd&i) Ul_®®lf6(#) <OT<S0<S5XS0 GurTSSTjn 5p(A) <SULSjSU65)LDU65)ud 
G)&rT®)d(3jLt). ^igjGsii Zero padding 6T65T^j ^65)Lpd&uu®dfB#>]. sjQesresflsb Gldjd#s>.i^Iuj 

(Lp65)JD#6lfl60 ^gJ/OTSiyaSlT (3)65)fDdau3 U®)LCiGuIT gl, Ul _ d>®65T rBrr65T(^ Qp65)60&6fil<Sp]li> 3—6TT6TT 
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idld&eb LD§)]uLi&m fid^uuueomd. ^gesrrreb 3 lfi^ss)auu (LfiGS)Gsr3i<sifle^l(i^r5§i rBLod^d 
®<s!n/_aa Geuessns^iu @eo (Lpddhud, paeueb&err @}65)udarrw6b Gurreu&.fi&rresr eumuuLi&eh 
2 _GfTSfT 6 $r. $i£6$)6$r£ £6)fi pu upfiarra ears^G^ Padding 3 t,(^id.. ^fi&esr ^peoid Qpstneo&sifleyjsrrsrT 

iSld&GV LD^!ULjS,6h fidfELlUldl _ 1760111), Uufifi$65T 6T6065)60lfil6pl6fT6fT frlfilUJ LO^UL/aOT 

fids,uu(fiGiD ^edlrr uufi]filfi( 0 j erfigefip ^LpuLfids 6rfiuurr§i.. 

Guifiaemi— 6T®£§]darrLlL!i-6b 9*9 ^essflemuj 3*3 emps.idLp Qpeoid emp&idQudGurray, 
@}6SU—d(&jLt) convolved output-esr ld^ul/ 7*7 eresru urrrjfiG^rrid. (ffi&yGeu uufijfilesr rBrresr 0 
pw65)60s,s7rl 60 !id (g^pem® eufiesi&aiefieb &i£hu iDShuLj&essm ^eiflfi^rreb 9*9 31essfliunesr&y11*11 
6T6ST LimrntDUULl® @l65)ud(3jlD Convolved Output-651 LDfilULILD 9*9 i.e [ (11-3) / 1 ] + 1 6T65rGfD 
< 316 S)L£>iLjLh. eresrGeu zero padding uiuesruQfi&yidiGurTaj], 9*9 ^essfliurresrg] convolution-d(§u 
l5Ijd(&)L£) 9*9 <3]6ssfl65)UJ 3— (TJjsi/ adso&>a6V (fifisji "samepadding" eresrjD Quiufieb 

<S>i6$)Lpd&lju(fi&l(D§l- (<fi6S)61J3i6h 6T6S)gU-ILD UUJ6STU(fifi^rTLD60 &l£IuJ LO^UL/aOT <570/6iyi£> 
euLprzianTLDeb, eyqF, uufies)^ 3 ]uuLpGiu uiuesruQfigieuGp "validpadding" eresrfo Quiufieb 
<3t6S)Lpdauu(fi)@)fD&i]. 


3i65)U@ uuy: 

Gwfi&^fihu convolution id fi op id pooling ifilqijLbufi filqtjidu rsesiuQufirpj s^0 uufifilesr ^sney 
3r(ri)d&uu(fi!dlfD3y. iSlesresrrj ft^is/juLL uudfileoishsn iSld&eb w§) u ipaGerr 3 >l u u ufiesip 
3ies)uiurrmid a rnsmi eu&fi&rrasr features -3t,& 3 i 65 )iDiLjid. (sfisyGeu 'Flattened array of input values’ 
eresTiyj 3t6S)Lpdauu($)ld}fD§]. £g) idmlfiluLj&QerrFC eresruu(f)id Fully connected network’-d^err 
Q& 60 idf>uuLd(f 3,es)i _ @ Geoiufiev eresresr uuid eresru&y euesiauuQfiguuQdlfD&y. 


difid&emu ertfrfi^daiaidLpeb eyqj) uuumesTgi (Lppefleb ereheimrry 2 _ 6 fT 6 fr&ji. Padding Qpeoid 
ui—fi!filfi( 0 j 6 T 606 S) 60 & 6 h ereiieuag] 3]65)iDd&uu(fiS>)rB&y. Convolution-d(Sju idlesresrrj uuid 
ereii Gantry iDntry&ilfD&y. <%6S)u@hurr3i Pooling Qpeoid uuid ereiiean^j &(njid](&j@}fD&y eresruay 
amduuuLLQeherrgi. Lofiro Q^idQeaaijd Gunesrrry CNN-go ^ShufieSlqijfi&y 2_06i//7a®a 

3HTlL(^61J§] 6T6STU&J fi£0 <^t,prTlLld@d SiLd(fi6S)!J6S)UJ fil&lfifi§)ld a ml (fi eu^fi 0 &U)LDrT(&jlD. 

6T6srG6ijprT6ST 3 l<sfT<S)fiG 6 oGuj 6T(fi)fia)idarTLl6S)L- rfifryfifild QmaemGuesr. Qurr&yeurra 
ejfiQmesrGea a_ 0 Si//Taa/jL/L./_ CNN iditi _ eba^esierr erQfi&y rsuidG&fijDeiJ my 3i$fi(3) umfifilu 

uufifidl <3j61?ld>3j]U UUJ65TU(f)fi&>ld Q&rT6YT6)jG^ @fDUUa3i 3] 65) ID IL/ID. 

httvs ://sist. sithub. com/nithvadurai8 7/5 Ocb753e 78bba477e3ef8c01ea338c63 
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import numpy as np 

import matplotlib.pyplot as pit 

import matplotlib.image as mpimg 

def zero_pad(X, pad): 

X_padded = np.pad(array = X, pad_width = ((0,0),(pad,pad), (pad,pad),(0,0)), 
mode = 'constant', constant_values = 0) 
return X_padded 

def conv_single_step(X_slice, W, b): 
conv = np.multiply(X_slice, W) 

Z = np.sum(conv) 

Z = np.add(Z, b) 
return Z 

def conv_forward(X, W, b, hparams): 
stride = hparams["stride"] 
pad = hparams["pad"] 
m, h_prev, w_prev, c_prev = X.shape 
f, f, c_prev, n_c = W.shape 

n_h = int((h_prev - f + 2*pad)/stride) + 1 
n_w = int((w_prev - f + 2*pad)/stride) + 1 

Z = np.zeros((m, n_h, n_w, n_c)) 

A_prev_pad = zero_pad(X, pad) 
for i in range(m): 
for h in range(n_h): 
for w in range(n_w): 
for c in range(n_c): 
w_start = w * stride 
w_end = w_start + f 
h_start = h * stride 
h_end = h_start + f 

Z[i,h,w,c] = conv_single_step(A_prev_pad[i, h_start:h_end, 
w_start:w_end, :], W[:,:,:,c], b[:,:,:,c]) 
return Z 

def max_pool(input, hparams): 

m, h_prev, w_prev, c_prev = input.shape 

f = hparams["f"] 

stride = hparams["stride"] 

h_out = int(((h_prev - f)/stride) + 1) 

w_out = int(((w_prev -f)/stride) + 1) 

output = np.zeros((m, h_out, w_out, c_prev)) 

for i in range(m): 

for c in range(c_prev): 
for h in range(h_out): 

w MyBM ftfa ui i n- 
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w_start = w * stride 
w_end = w_start + f 
h_start = h * stride 
h_end = h_start + f 

output[i, h, w, c] = np.max(input[i,h_start:h_end, 

w_start:w_end, c]) 

assert output.shape == (m, h_out, w_out, c_prev) 
return output 

img = mpimg.imread(/cake.JPG') 
print (img.shape) 

X = img.reshape(l,142,252,3) 

fig = pit.figure(figsize=(15,10)) 

axl = fig.add_subplot(2,2,l) 
print("Shape of Image: ", X.shape) 
axl.imshow(X[0,:,:,:]) 
axl.title.set_text('Original Image') 

ax2 = fig.add_subplot(2,2,2) 

X = zero_pad(X, 10) 
print("After padding: ", X.shape) 
ax2.imshow(X[0,, cmap = "gray") 
ax2.title.set_text('After padding') 

ax3 = fig.add_subplot(2,2,3) 

W = np. array( [ [ -1, -1, -1], [-1,8, -1], [-1, -1, -1] ]). reshape( (3,3,1,1)) 

b = np.zeros((l,l,1,1)) 

hparams = {"pad" : 0, "stride": 1} 

X = conv_forward(X, W, b, hparams) 
print("After convolution: ", X.shape) 
ax3.imshow(X[0,:,:,0], cmap='gray',vmin=0, vmax=l) 
ax3.title.set_text('After convolution') 

ax4 = fig.add_subplot(2,2,4) 
hparams = {"stride" : 1, "f" : 2} 

X = max_pool(X, hparams) 
print("After pooling :", X.shape) 
ax4.imshow(X[0,:,:,0], cmap = "gray") 
ax4.title.set_text('After pooling') 

pit.show() 


(142, 252, 3) 
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Shape of Image: (1, 142, 252, 3) 


After padding: (1, 162, 272, 3) 
After convolution: (1, 160, 270, 1) 
After pooling : (1, 159, 269, 1) 
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14 


6£0 ui_d,6$)&,d Qarr($)£§i <3l§)l<sv ^_efrerr§i rpmurr? L^ssxssrturr? <oT<s$r asssfldad Q&rrsbsyjGiJgi 'object 
classification' <oT<sstid rrrsb, ui_fi§)l<sv p_efrerr spsuQsurrqrj Qurrqp^fpih (areirQssrsirssr ctsst 
3j<sssf)d&& Q&rT<5V60i<siJ&,i 'object identification' ^(3}ih. GeijissxsvaesxsiiTd Q&iucuppQaesr 

ajpQassrGsu uuSIp&l Qupp AlexNet, ResNet, VGG19, InceptionResNet, GoogLeNet, DenseNet, NASNet Gunesrp 
iDrri—abash ap6S)puS}ab 2-shsrrssT. ^jsupfilsb ^rprTsuQprTasrssypp C oprjpQpfjjppi pLopi ppoyat^paGapu uuSlpd) 
^dsiflppi uiuasTuQphild QaashmsorriD. 


ImageNet arasrupi aussyaoppanp^lsSlqpppi ®T@d5d5UL/L/_ uabGsupi ui—miaamand Qarrsmi_ s ^ qt , database 

^jpi s^suQsump auQTji—(Lpib ILSVRC (ImageNet Large Scale Visual Recognition Challenge) arasrp rglafisdlatn&ir 
pi_ppi§ilppi. fgfippd acurrsSlob (2 iDpan.fil uj iDmsbaah ^asiasrppiiD umi(p QupjSlasrpasr. ^jasisuastflasr 
pisbsShup past as) id a&) uj 2 -p]filu($)pp ^piQurrasrp osurrsbash a^suQcump 6U(irji_(ipiD filaipppuuQSilasTpasT. 
6T6 st(Sg)J ^jppas)auj iDmsbaafilsb fpprrsuQprrasrsstp arffippi purarp (Ssusmipiu ui_pas)pd QarrQppi 
aassfldad Gorrasrasmsb, ^pasr aamsfluq Qurrp] surra afhurrapprrsir ^j(pd(piD. 
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14.1 Computer Vision 

n5L£>§] &6$$fl<ssfld(& ) s^0 ui—gempu urrijp&ii ^idlepieherr sfilstyUJiu&m <oTssiQsst<5sksst stsst 
<gj(SS)i—UJmsfTLju®£gd aifDQjd GlaurQuuGg computer vision <oT<sstq] <s>i(SS)Lpdauu®LCi. 

£§)<$<$fflnatzj object detection- <s<* mssr Gsusshsosmud Q&iusuG^ YOLO 6T65)iLD algorithm ^ 0 /A You 
Only Look Once sresruG^ YOLO stsst^j ^65>Lpdaiuu(fildlp§]. £g)g/ 6£0 uu^65>^ 
srQ^gjdQ&rrsm®, < 3 ]£S)^ dlrrp dlrrp gjsm(fs.snrrs, grid-so u^sflsv ifilflddljD^]. iSlssrssrj 
tgsuQsurrqtj grid-sir lS^]ld Image classification lcjd^Jld Localization <oT<ssru&ji rsemi—Clu^jdljD^]. 
iSlssTSsrj sysoQsorrq^ object-d^umssr bounding boxes-go asssflgg], sysbQsorrq^ class-dg^shi^id 

^ss)LDSu^rD<3irT<ssr [§}<9iLp^3iss)suiLiLb asssflddlingi. s^Gp object use ^urd&siflsv lSssst^lL 
id sm (find asssfldaiuu®6iJ6S)g£ gsfilrjuug(D(&j bounding boxes-go ^jesrestnd §]sbsShuLC>rT& 
^jsfi)L£)d& 2 _< 56 iy 6 uGp Intersection Over Union ld fiiiy id Non-max supression Guitsstid sfilsytUJfRi&SYT 
^0^ 


Face Recognition srssTug] gfoGurTgi srfii^id upsoeorrau uiu65ru(fi)^uuLL(fi) soq^dlssTro s ^0 
aSlsqtlUli) <Sj,(&)Lb. <5f >611 G) SUITQJj 611 ifl 65T 65)S,ljGudlufl60lld 3k/_ ^gJDarT6ST app ashsng]. £g)/5<5 
G6U6S)6C6S)Ujd Q&LUsn^/Dfi) YOLO $l£6S)6tsrface verification LDiprgjid face recognition 

6T6$r ^pemuiT&u iSlrflfgiu L/flfad Q&rrmmevmd. rBLD&j etn&GuffiuSleb o^sttstt^I Gurrso 
^ 0 Si/ 0 S!n/_uj (Lpagemgd anrsmiSlfig] ^jesresurp^i (tpaum ^sbesieoiurr srssrd sag] face 

verification ^(fijid (binary classification - one to one mapping). 


Face recognition 6 T 6 sru§] s^0Si/0S5>luj (Lps,d, 6 S)^d Q&rrQggi ^l^esiesr db-eb o^shsn uscGsogp 
(Lpaihj&qsrpuesr spuiShltfi) srp^ (Lpagg]uesr G)urT(njr 5 §j&}fD£>] srssrd aemQiShq-Uuaj] face 
recognition ^,0/A ConvNet sresrugj Gsuemeoetniud Q&iu&imspiid. 

^ 61/(o) 61//T061/06S) Z_ UJ (Lfi3i^^ld(^LDrT65T 6£0 LLIT^flU UL _ d>65)3) 6T(fifi^]dQ3ifT6m(fi) 

G&L&dd)to&)i. (sfjatfGsu 'One short learning’ stsstiqj ^65>ipdaiuu(fi)id. ^jLd(Lp65)fDufsv 
G&LSldtajidGuTTgi, spfDQ&ssrGsu G&Lfd&uuiltfishsrT it, u if sir (LpaGLD ld/dCIjditq 5 (tpeai/D 
surs^rreb 3aL < 3 i§)sv &tD!Q] um/gjuLii— features anrsmuuL^ssr <3i^65>6sr ld/dQ/dit^ rsuprra 
G&l£ 1 dSlip§ 1 . 6T65tG6u "^6op ^/Tfflfr ^6op " stsst ^fUurrs, aessfld&syid (ff^svtssrfi 

£6filijuu£rn<3;rT3; eurs&Gg Siamese Network ^(&jih. 
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Siamese Network (oTSSTUg] similarity function-g)u uiU 6 STU®£$ 5 l Gsusnsosmud 

Gl&iu&lfDg]. £—6fT<oi?i _/t<® suQ^SlesrfD ui—£$ 5 lfD(gjLC> db-sb GaLSldauuLLQxshsn 

ui_^S>lrD(&)LDrT<5KT Gsurrpurufss)L_ (Lpgsflsb asssflddljngi. iSlssrssrrj Gsu/gjurr® srd,^ ^gi/OTSiy 
susmrr ^&s)LDUJ<svrrLb <oT<s$TU£to&mssT threshold-^ ^ss)i£>dd)rD§i. Qsueifluu(^)^§]SlssrrD 
Gsunyurr® threshold- <£0/i> (^stu/D surra ^(njuiSlssr "(Lpaid Qurr(r^rB^jSl/D§]" STSSTSifif, ^^lauma 
^IQTjULSlssT "(Lpaih QurTQTjrB&Gtflsbssxso" STSsrsifii) asssflddlro^]. £§)<$<$ ssisuj (LpsnjDuSlsb 
0si/0S3>i_uj (Lpaih aGssfldauuQidGurTgj GjiDu(f)dlssrp ^Lpusmud asmi^rfhu 2_<$<5iy6i/<3<$ 
triplet loss function ^0/A Anchor img stsstu^i Qarr(f)dauuif(f)shm ui_ld ld. 

positive negative images srsiruaj Qarr(f)dauuLl(f)shsfT ui—fajisir QurrrnyfajdlssrrD Ldfo^jii) 
Qurr(i])r 5 & > rT& > ui—flask ^(a^ih. anchor image, positive image, negative image srssrfrp 3 UL—&>6S)&> 

g^uiSIlL® ^jLpusszud asmd£l@£)in§i. (oTssrGsu^rrssr triplet loss stsst/d Quiurflsv 
^ss)Lpdauu(f)S](D§i. 
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15 Recurrent Neural Networks 


C97T<$/7776W / Q[5 L.G) SUnijSl<50 61V0ii> £g) IJSSST® ^shsffl^Q)d, ,$/761/<S6Yr 

s^ssiQfDrrQL^nsiiQ] Q^m^piSlsbsomDsb ^(njd(&jLC). <oT@£aj]dairTLLi—rT& s £0 sifLi^_@ys!D/_uj egijj 
^uj- s^hsuu^ein^u QufQfQidQ&rrtsm® <S 4 s><s$t s^esxscemujd &6ssfld(3jLC> G&rrpissxssTisnuj 
6T@£&fidQ<3irT<sssri—mob 400 &3jHJ <31iq- s^lIl^id^ spqij aShss)<so<ss)UJU-iLD, 600 &§ijjLq~ 

6£0 S)S]ss)<Mss)UJinLb a<sssfld(&jLt>. $l[5& ^rr<sm@Lb speirGiDrTQi^msir^j Qprri—ijLQsbsorTp 
aS\<surrthjai<sh. 600 &3,pj ^iq-d&nssi aSltssxsotsvnud aessflda s ^0 QrpLLGhsunijd ££l&/D(&j (Lprsempiu 
400 &&>pj <S4i4- qS'lLls^jdi^ mssrssr s^lesxscemujd ^essfl^G^mb <oT<sstu<ss)$ rffesxssre^lsc Gsxsuggid 
Q&msrrGrr£ G^ssxsuuSlsbesxsv. £g)g/ Gumssr/o spesrQrDrrQi^msir^j Q$m—iji3<d)<son$ 2 _GrrGi?Ll®£ 
^/rsi/< 95 (sy 5 <s 0 &rr8>nij65tiT QrBL-QsiJrTijGtnau uiu sir u ($)&>&> so mb. 


tS^stsmsv d\<sv &LDUJ[d&<si?l6b s ^0 s^ls^iu^Gm^d a<mfluu£jD(aj m_rfddl iu rrs, suq^dhssrjD 

^shsfTL^Q)^ ^rjGLj&Gh 'S46$)6$t^§}lL spssrGjnrrQi^rrssTfrp Q<$/tz_/tl/63>z_izj<$/t<* <s4Gmr>d)<sirin6$T. 

67 @^§/m/tl @<50 urhi(&jd &r 5 <ss)£aGifl<sb aGtisfldaiuu(S)L£> <sp([i) ufddlssr oShsmsoiurresr^i <S 4 ^&$t 
gfoGumsinpuj GfiUssKsvsmuu Glurrnpjpg] lolL^ld emunumneb ^&JD(3j (Lpssrssrrj ^gyss n£>pg> 
oShssHSO&srrhssr ulLis^ luesxsvd <ssmddlsv Q&rrsmGL- &6ssfldauu@d)jD§j. <S4g4 GurreoGsu s £0 
Q&rrfnQfDmsmrrd G&lL® aessflssfl mssrosT LDir^Jfliumssr u^Jesxsc <s 4 <si?ld& Gsusmt^id mssru^i 
QsufruLD tspQij Q&msb<ssxsou Q u nrgjgsji ldlL@l£> ^emmumDsb, uebGsu^j Q&rrjnaGiflssT 
G)gm—[jdd)<ss)ujd OcS/tswCSl &6ssfld&uu(b)d}fD3)i. ^]§]Gurresr^j ^ssnAiLjib gjjGiiaGen 
'sequential data' mssr^j ^GnLpdauuQldlrDgl. ^)d>ff><s$)&UJ &aj<sqash temporal ldjd^JLD time-component 
ggu QufOjrp <sfi)<smhi(&jLb. ^j^iGurresrp ^psLjs,ss)SfTU uiussruf^^^l &<ssrflun&iG$)6fT 
r£}&Lp£§iGiJgfD(3j sj/dfO sij ss)3iuS )<m RNN dipuurrssr siJiq.siJss)LDUss)uu QufOjrp <s^l<snrd(^dlfD§]. 
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^jgSST <SULSf.SU6S)LDULj S^QfJ S£0 rflgj^prTGStGSrd (s)«/7«L|/. (LpgsSlsi) GUQTj&tsirfD gtjaSl<S5T 
^Lq-UUistni—uSlsb ^ 3 lgg rflu^rjmssr G)airT®d(gLC> Qsu&fluf® G&LSldauuiL® <3l®gg1 GiiqTjdUssTfD 
gpsi^i^ssT ^j6S)6md3iuu®Spg]. lSUsstsstij ^GSiGuufpGssn^Gsr 3]uyuuGifti_uf6V Sss)i_d(gLb 
Qisusifluf® G&LfildauuLL® LSsm®ih rSh®gg®gg] <su(gS<sirin gijGif&ifsgissT QgrTi—ijddlujrT& 
^6n6mdauu(®S6STfD6ST. a<ssuSujrT& ^jGsxsuansiflesr ^iLq.uues)i_uSlG<scGuj aessfluLiasrr 
rff35Lpgguu®6UgrT(sb ^jg] Recurrent <oT<sir(D Quiuflsv 3]GS)Lpds,uu®Srngl. s^sijQ<5urT(g 
(Lpem/DU-iLD S<ss)i_d(&jLb gfD&rrsSl&i Q(5U6rrliif®<3i6ft6fT G&Lfigg] GSiGudsi (gfigj ^_<sh^STgp Gp(g memory- 
ggu uiUGsru®ggjS/DgJ. RNTN gt GST in Qgil(ol<surT!jS<sb $l[5g memory-d(g LSTM (Long Short Term 
Memory) GTGsrrrjj Quiurr. ^jgg (£}i—g5>l<sbgrT6sr gtgstQgstgstgst aShsyiUJihiaQGfTsbeofTLD <ffg]6UGS)ij 
G&L&dauuLlChlsfrsfTssT 6 tgs)Ild (ftSUsurnh s^sijGhsiirTtg (LpGSifoiLjiL amso-Gpijg^ssT ^LSf.uuGfti^uSUsb 
G&LSldauuQiSjngi. 


^lg]GuiT6STQ] Qgrri—ij rff&LpGy&GSiGYT r£lGS)GST6)dlsb ssxsuggid Q&rrGssr® Liflggi G)&iu<sbu(®6iJgfD(g 

GjfDfD GUGiftSiiu itgst SLi_ GftLDUGftuu Quinn)] G^GfTfdtgGUgrrGV 'Text Processing' GufTGSTfo 

Q&lLKSOaSlflsb (sfjGS)gU UUJGSTU®gg<50rTLD. <3igrrGUgl Gpcg G&HTUlS]6V 3]GSUDrB®(gd(gLD 
356S)g356S)snGujrr, &LL®6S)rja6S)6nGujrT GlarrQlgg] RNN-d(gu uuSljnSl ^siflggrreb, 3 ]g] 

<3IsijGurrijGftgaisiflGST QgiruijLi u u Lq. uuSlrn& gs) luu Qutn^jd QamshSfDg]. LSlesresrij Lj®grr& s^(g 
GorrrjgGS)gGsnud Qsnr®gg], ^ssigg Qgrruijgg] r3lsftLftiuaSl(gd(gLh simijgsftgssnu asssfldad 
Q^fTGSTGSTfTGV, g IT GST (olUID^jd Q<3HTGSSTU U l£lfO & lid GST UUy 0«f7@i«UL/LL Gil rTfjgSftgU-l I—6ST, 

3]®S, QglTL _ [JL]GS)L _ UJ GUmjd > 6S)gS,GTT GTGStQgSTGSTGST 6TGSTUGS)g &GSsflgg]d Q&rT6V6VIL£>. GTGStGgU 

<5f>(IJ)(o)JiflGST Qud<SS)&Ll l-lflggl QaHTSTTGUgl, ^gjnGsifDU U®6V <3]61?luUg], QLftlTL^lLdrTfDfDLCi 

Q&LUGUg], uscGsutryi (gpefa^GSim ^rrmiiGog], ^sftijujrTusbasftisrT rfl&LpggisiJgi Gurrssrjn 

G(5U6S)<50<ai^gd(g RNN-^u uuJ6STU®gg<somi>. 


RNTN GTGSTUgj RNN-GST Gy(g GUGS)<9i. GlD(6<3^ft5hU <3IGStGSTgGSlglLjld QsU^JLD ^gglhlSsO QLD[TLfld(g 
ldl1®lc> Q&LLnurTLDGV, GSTgst iSlfQ QLDrTuSl&Giifl'SVi ld Q&iiig] 3iTTLL®<sugfn(g NLP (Natural Language 

Processing) GTGsrgy] Quiurj. ^gnsug] <syg QiDirLflufspiGhm QuiurjdQ&rrsb, oShssxssrdQ&msb, 
g-fldQ&rrsb, ^jGStGssruLjdQ^rrsb GurT6STfn<sufD6S)fD 3 ]GS)UUJrr 6 mT amsssyigtsb (Parts of speech 
tagging), <sp([i) Q&msbeShsb £g) (gd(gLb Gsuijd Q&msbssxso <6T®gg<sb ldjd^jlL 3 ]gGS)Gsrd, gqgisfi} 
GUQTjSlGSTfn iSlfQ Q&rrfn&GSiGiT 3 ]GS)UUJrT 6 fuL sir TGmi gGV (Stemming & Lemmatization), Q&rrjn&siflGST 
uTT(gurr®(segmentation) GurrGSTfo Gpcg QLDfTLflGsnuu urorfilu Lpflggi Q&msrrsrrg Ggeftsuiunssr 
3 ]GS)Gsrd,g] 3 ]ii)<9 : ii]<9iGS)GYUL ]ld RNTN Qu/ngu oSl<snihJ(gSg)gi. Multi-layerperceptron 6TGSTUGS)gu]Lb 
^][Bg G>6iJ6S)c5odarTau uiuGsru®gg6vmi). ^gstitsv ^jggRNTN (jfGSTGSinl) &jD!Q] GldlLulIl- 
^Lb&rhi&qiriyi—GST (*£i6i]G(SiJ6S)<506S)Ujd dljnuurr&d Q&iuSjngi. 
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16 BM, RBM, DBN Networks 


Boltzmann Machines <o7WL/§i)<siS)0/5^/ 2_06iv rnssrG^ Restricted boltzmann machines (Lp&^sSlsv 

Boltzmann Network <stsst/d rrsv <oTsstssi ioTgst^j urT/juGumb. LnrrSiliflg ^/ rsiyasyfisi ) 2_ehm 
^l^laiuUL^UJrTSsrfeatures-^) £g) 0 / 53 i/ /5LD«0 Gsussstl^uj (Lpddhu ^Lb^dis.osxsn ldlL^ild 
2_0<si//r>*0/i> GsutssKSVemu Boltzmann Machines Q&iudlrD&n. £g)g/ Qsu^jld input LbfDrrpih hidden 
Gsoiusnij ldlIQld Quforgj <sfi)<SfTrhi(&jLb QrsNQisiimjd ^0/A ldjdid deep learning LDirisfleb 
2_<shm§] Gumssr^j output Gsoiurj <oT<sst(d spssrrrg gesfliurrad dUssu^iurTg], )rs&> Geoiuflsb 

2-<sh<srTnodes ^ss)sst^^ilci undirected (LptsmjDuSleb ^jQsxsmds.uuLdLq.q^di^Lh. ^^rrsug] ldjdjd 
QupNQsumjd Gurrsmirp ^/rsiyasmsir (LpssrGssmddl ldlLQGld Q&spid&jmixsb. ^<su(D( 5 l<sb 
2_06i//T<a«UL/(5)/i> gjjGii&err ^GSxssTg&n dhsm&uSl'SViLb Q&<svid&>uuLlCh) lljbjd nodes-®o 
2_0Si//Td50/i). toTGsrGsu^mssr $]§] 'Generative Deep Learning Model’ <oT<sst^j <s>l(SS)Lpd&uu($)d)fD§]. 
^j^sir ^svnDULj iSlssrsaq^LDrrrru. 


BM 
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3 ^ mm spdjQmnqij nodes-Lb %l/l/ ^ib&rhi&s&md aemi—rfliu 2_<56iy/i> s^0 feature 
detector- ^,<s Q&iusvuifiLb. {gfi&ilsb ^ssxsssTULj (LptstnjD lorsitsnrr^j ^(njd(&jGlLD<ssrjDrT6b, ^ussstQ) 
Gsoiuflsb 3_mmnodes -ld sysKrGjnrtQt^rrssTgp ^jss)sssruss)u <sjfbu(ff^ld QgurmsiJgiisiT, syGtj 
Gsoiuflsb 3—mm nodes- ld prdangrijdt&jm ^jss)smuss)u <5jrbu(fi^^ld Q&msrnQrijLb. ^gssmsb 
2_06iv/T<sd5 g^-isf-iLi <3]§>l<9i ^gj/syrsi/ feature detectors-QrBifQsomjSlsir (Seva ld (3j6S)fD[5§i 
&msmuu($)Lb. pGfilrjuugfb&rra sof^G^ Restricted Boltzmann Machines ^ 0 /A £§)< 5 <arr 

^SSHDULf L5?S5TfiU0LD/Tffy. 


RBM 



RBM ioTsstu^] spGp Gsoiufleb 3_mm nodes ^fdaii^d^m ^smsmusmu sjtf>u(f)£§i)d Q&rrmsijtsng 
LDiftfub restrict Q&iuiLjLb. ldjdjdul f £g)^/si//i> BN-®dlj GumssrGjD ^fjrrsm® Gsoiuemud Q&rrsmi— 
flumps!) QrBifQsomjd ^(^ld. (Lp^svrrso^] Gsviurr input stsstsl^ld, ^jrrsmi^rrso^] Gsoiurj hidden 
<5T<5KT<5LIld ^stnLpdaiu u ($) ld. spGtj Gsoiufflsb 3_mm nodes-d(&j<m 5jfDu(f)Lb ^jes)essrues)u 

ldlLQld restrict Q&iumprTsb, 'Restricted Boltzmann Network' 5T6$rjB Quiuflsb ^smLpdauuQdlrDg]. 
£g)g/6iyLD undirected srssTU^rrsv, iSlssiGssmd&lu urrsL/^sv Qp sold gradient descent (LpsKi/nufsb 
^m6LK[ijda6S)m&fl Q&iuGug] <oT6stu§] $l$ 5 l<sb dlsKiL^iurrg]. ^gV^/D0 udhsorrs, 'Contrastive 
divergence' 5T<5S)iLb (Lpstntn uiu5Kru(f^uu(fdlfD§]. $i§i ^fDtnstmsod (^es)puues)^d 
(grfildG&rrmrr&d Qarrsm® Q&iusbuQLb spqr, ^LufiShum ^ldsstuttlLlp&sxsst snsuggi 
Gl&iusbuQdlfDg]. 5TsstGsli ^§1 'Energy based model' SLiGtnaemujd &rrijr5&&i] stqstsowld. dimensionality 
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reduction, collaborative filtering, feature learning and topic modeling Gumssrp tit & Gift <sb Q&UJGOrrprfil 

su0ffl//)ai/. RBM gtgstli&j visible nodes gul£1G>uj GUQjjLD ppsij&qsriji—sir random -^ad, 

QppfQpffidauuif l— GfTGLj(!])&,3><s$)m(weightsj ^j&S)hidden nodes-go a_0Si//7<S0LD. iSlesT&srrj 

2_0 si// 7 c*<suul 1 l hidden nodes ^mstK^daGnmij uiuGSTLKjfpS) GT&skr&vsfld&nauflGb visible 

nodes-go LDtrp 2_0fiu/r««LD Qffiu&lpg]. 


Deep Belief Network GTGStUGtnp (s^estsstiLb a=ppi GunliuQfppuutEt^ RBM GTGsrsotTiii. ^jpi Stack of RBM GTGStpi 
<2IGS)Lf>dauu(i)i&l[oaji. RBM-sst speitQeiJtTQij GGCiufilGpiLD S-ertm pssflppssfl nodes-est ^issxssstuqaGsxifTp pafilpppi, 
LDtrjDtrai Gsouj(r^dSlss)i—GuJUJrTGsr^iGS)GssruGS)u gendgjjGfildSilppi. ^jpi G^GStpidtpifi GuopuiLt— hidden 
G<souj[T<9i6S)GtTd QarrGssrisi.(pij l 3 gst, GgGijQGurT(p (3<souj([ijii) p<o&rd(3j (Lpponpuj GGOiumi—Gpiif. pasrd(p 
Gsmumi—GsiiLb ^GS)GmuGS)u <sjpuQpGild Qamsssr® Q&uJGbuQSilpgi. pond(p QppGmpiu GGOUJ(pd(p hidden 
(3<souj[TrTaGipi), pasrd(p ^®pp GGOUJ(pd(p visible GGOtupnaG^ii Q&uJGbuQSilpgi. GgGrr GGOtu(pd(pGrrGGtrrT 
^<sb<so§i ^(T^<xrT6S)iDu5l<sb ^<ss)LDiurrp Geotu^i—Gesrrr ^jes) emu emu GjpuQpprrgi. ^ipGst ^gsilduli 

lS\GGtGU(JTjL£)tTp]. 



O 

0 



0 


O 

0 


O 

Input 

O 


Hidden 


GiDps,GS!STi _ ^l&sy&STfigjLi) UPN(Unsupervised Pretrained Network) Gusnansniud &rrijpppi. ^prrGupi ppQa&srGaj 

unsupervised (Lpanpufsb uuSlpdl ^Gtflppi 2_(njGiJ rrdau u lLl_ algorithms ^0LD. RBM GTGsrupifeatures-gou 
PiliflpQpQppi prjGijaiGsxsfT lBgsstQld Lop]S-(pGurrdaLb Q&iliuju utuGsruffilifi. ^ssmeb au^GstLomsst 

features-god &S)3>iurr6fT (Lppujrrpi. ui—tEiash, spatfl, spsSl GutTSStpeupifilssr Lfipi Q&iusbuLfff cluster images, video 
capture sound, text Gutrestp Gaj&ftisva&nmd Q&iuGuppgtj DBN Qufi&jih uiuGstuQdlppi. 
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dipdasssri— <at(^p§idaimfis\.<sb 943 uiussrrjam, 1682 ui—iEia(§rijd(&j euLpiiiSliu ppeu/flss)# Ld^luqasfr uu5}pff\d(&), 
u/fKS&rTgGtnesrdt&j gt&st rjsm(f GarruL/aemra sshmssr. ’ul.base’, ‘ul.test’ otsm/ld ^rrabsr(f Quiufisb ^) 0 c*@ld 
^jsupesip difidaessri— (LpasufluSbsb Q&eirpi utflaSlpdaLb Q&iugjQamsrtGfTsomb. 

https://www.kaggle.com/prajitdatta/movielens-100k-dataset 

^pppeyastflsb s^shsrr 1682 ui^dia&nsmLiLb pesflppesfl features-^a torQpgidQamsNsr® , pm!) RBM Qpsoib 200 
features -go (SpijrBQpC^daijQurTSlQpmi j. gtgstGsij RBM- sir sstrstfif® (Ssoiuiflsb 1682 nodes-ih, hidden Qsoiuflsb 
200 nodes- id ^ss)LDUjLb. g^rsp ^strstfif® (Ssoiufleb ssfren e^eijQsurTq^ node-tb <s^eijQeurT(^ ui_p<ss)pd (gnfld^ib. 
^jpesr euiflQujprTGST QiDtrppib 2_6ww 943 uiuesrijai^ib ^d(&)t$lui3LlL- ^siflpp ptreuffles)# 

LoSjluLiaerr Q&spippuuQib. iSlsir&srtj sampling Qpsoib 200features Qp:jfQp(fd&>uu(f\Lb. g^ppamsst rflrjsb 
LS?65TSU0LO/77p/. 


https://gist.github.com/nithyadurai87/74b40cab9cf65929c6felc99clb58a9c 


import numpy as np 

import pandas as pd 

import torch 

import torch.nn as nn 

import torch.nn.parallel 

import torch.optim as optim 

import torch.utils.data 

from torch.autograd import Variable 

def convert(data): 
new data = [] 
for i in range(l, 944): 

id_movies = data[:,1][data[:,0] == i] 
id_ratings = data[:,2][data[:,0] == i] 
ratings = np.zeros(1682) 
ratings[id_movies - 1] = id_ratings 
new data.append(list(ratings)) 
return new_data 

W = torch.randn(200, 1682) 
a = torch.randn(l, 200) 
b = torch.randn(l, 1682) 


def hidden(x): 

kUHiU^U 
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activation = wx + a.expand as(wx) 
ph = torch.sigmoid(activation) 
return ph, torch.bernoulli(ph) 

def visible(y): 

wy = torch.mm(y,W) 
activation = wy + b.expand as(wy) 
pv = torch.sigmoid(activation) 
return pv, torch.bernoulli(pv) 

def train(v0, vk, phO, phk): 
global W,a,b 

W += (torch.mm(v0.t(), ph0) - torch.mm(vk.t(), phk)).t() 
b += torch.sum((v0 - vk), 0) 
a += torch.sum((ph0 - phk), 0) 

training^set = pd.read_csv('./ul.base', delimiter = '\t') 
test_set = pd.read_csv('./ul.test', delimiter = '\t') 

training set = np.array(training set, dtype = 'int') 
test set = np.array(test set, dtype = 'int') 

print (max(max(training_set[:,0]), max(test_set[:,0]))) 
print (max(max(training set[:,l]), max(test_set[:,1]))) 

training set = convert(training set) 
test_set = convert(test_set) 

training^set = torch.FloatTensor(training_set) 
test set = torch.FloatTensor(test_set) 

training_set[training_set == 0] = -1 
training_set[training_set == 1] = 0 
training set[training^set == 2] = 0 
training set[training set >= 3] = 1 
test set[test_set == 0] = -1 
test set[test_set == 1] = 0 
test_set[test_set == 2] = 0 
test_set[test_set >= 3] = 1 

for epoch in range(l, 11): 
train_loss = 0 
s = 0. 

for i in range(0, 943, 100): 
vk = training_set[i:i+100] 
v0 = training set[i:i+100] 
ph0, = hidden(v0) 
for k in range(10): 

_,hk = hidden(vk) 
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_,vk = visible(hk) 
vk[v0<0] = v0[v0<0] 
phk, = hidden(vk) 
train(v0, vk, ph0, phk) 

train_loss += torch.mean(torch.abs(v0[v0>=0] - vk[v0>=0])) 
s += 1. 

print('epoch: 1 +str(epoch)+' loss: '+str(train_loss/s)) 

test loss = 0 
s = 0. 

for i in range(943): 

v = training_set[i:i+l] 
vt = test_set[i:i+l] 
if len(vt[vt>=0]) > 0: 
h = hidden(v) 

_,v = visible(h) 

test_loss += torch.mean(torch.abs(vt[vt>=0] - v[vt>=0])) 
s += 1. 

print('test loss: '+str(test loss/s)) 


[finsvidarr&ST siSlendanh: 

1. QppsShsb LDtr^lflp ppsyaerr trainingset, test set GT&niii) Quiurflei) 2 dataframes-^s, GffuSlasuuL® LSlesresrp 
numpy array euipaSlsb LDrrppuuQSilGSTpGST. 

trainingset = pd.read_csv('./ul.base', delimiter = '\t') 

testset = pd.read_csv('./ul.test', delimiter = '\t) 

training set = np.array(training_set, dtype = ’int) 

test set = np.array(test_set, dtype = 'int') 

g-ernstr LO^UL/assrr L^GSTGUQTjiDrrp] ^GS)im)Lb. 

print (training_set[:3J) 


[[ 1 2 3 876893171] 

[ 1 3 4 878542960] 

[ 1 4 3 876893119] 


[ 1 5 3 889751712]] 


125 






(Lp^si) column-eb ssnsnaji 1 (Lp^sb 943 gugs)[t ^igs)lotb^ LHUGsrQTjdarrGsr GTGSSTGStGssrd (gjjfildSDjogi. 

2 gu§i column -go S-GtTGn§i 1 (LpgGO 1682 gugsitj ^gsilotb^ ui—rRiaignfdarTGST gtgsst ^(apd. ^igrTGUgi (tp^GO 
uiugsttI GTrsQprsp ui_iRiaignjd 0 pijGuiflGS)# LO^UL/assffxsiT ^(^sg/snwm/ gtgst QouGufluu^^tui^Gsr, 2- 
gu§i 3-gu§i gtgst <5j>eijQff)jn(iTj uiuGsr^id rating ui^diaesim QGUGfiluu(]>i£§iid. 

^ 3 Gu&ji column -go s^0 uiuGsrtj <g\d(ii)tfi\ui5liLi— ui—£gid(3jd Q&rTQggiGrrGtT ^pGuflGsya gtgsst ld^Iul/ 
<S4GS)LC>r5Gil(njd(gjLi>. 1 qpgGO 5 gugsiij ^gshd^ GTGSsraGtrrTGb GpQtj ui—^^Igst gpid (ajrfild&uuQSlrDgi. 

^ 4 Gu&ji column gttb ts Qrsrj&tBlGb ^luuiuGsrrj grj LD$£luGS)ud QaTT<f\^§iGfTGnrTij gtgstugs)^ 

QGUGifluuQtBgiSlrDgi. 


Qumppid gtt£tJ)GS)gst uiuGsrrfai^id, ui^mian^id 2 -Gttgttgst GTGsmd GdlGupid i3gstgtjq^i£itt^i 
s,Gssr®iS}i<pd&,L3uLl®mmg]. GjQssTGsflsb ^pprjGgaiGn training, testing gtgst £g)/76Mn_/735/j iSlflrB^d^uu^rTGO, 
^TjGssrispGgiLt) S-Gttgtt Qutfliu Lo6j)uL/ atGssr(fii3isi. danduiL®, ^SilGO QuijUugi atGssr(fii3isi.daiuu(fiSiljD§l. 

print (max(max(training_set[:,OJ), max(test_set[: ,0]))) - 943 Users 


print (max(max(training_set[:,lj), max(test_set[: ,1]))) -1682 Movies 


2. ^uGurTgi RBM g<s$T§i Ggugsxsogsuu^ Q^/T/_/570si7^/D0 GTinrDGunrpj 943 rows ldjd^jld 1682 
columns-eb <s4<ss)LC>r5& &>u sufles)^ L£>^!LiLj3i<ss)<snd Qamsmi— s^0 ^sssfl £_0si//t<s<sl/z_//_ 
G<5uem(^)LCi. <^£$<s$)&iu ^gstlouiSIgstgst 2-(nj<siirTd(& ) L£) Gsugstgogstiu^ttisst convert() function 
Q&iu&jng]. 

def convert(data): 

new data = [] 

for i in range(l, 944): 

idmovies = data [:, 1] [data [:,0] == i] 

idratings = data[:,2] [data[:,0] == i] 

ratings = np.zeros (1682) 

ratings [id movies -1] = id ratings 
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new_data.append(list(ratings)) 


return new data 

®l[5& function-d(3)<sh (Lpgsflsb epqrj &msS)u uifLq.ujsb ^(r^surrdsiuut^dlp^]. LSleiresrijfor loop 
qp<sol£> spdjQsorrq^ uiLKSSTqjjLD STrsQ^rs^ ui—rhjaq>njd(& ) rating ^j<sdlf^]<shmrr[j, loTesrGhssrssresr rating 
^eifldffiiehenrTrT <oTgs)ilL aShsutjih id movies, idratings OTgo//i) (j^rjsssr® uiLi±uj<5£)&<Sfm& 

G&l£ 1 dauu®£) sstjdsst. iSlssrsuq^LorriQ]. 

print (id movies) 

[ 2 3 4 5 7 8 9 11 13 15 16 18 19 21 22 25 26 28 

29 30 32 34 35 37 38 40 41 42 43 45 46 48 50 52 55 57 

58 59 63 66 68 71 75 77 79 83 87 88 89 93 94 95 99 101 

105 106 109 110 111 115 116 119 122 123 124 126127131 133 135 136137 

138 139 141 142 144 146147 149 152 153 156158 162 165 166 167168 169 
172 1 73 1 76 1 78 1 79 181 182 187 191 192 194 195 197198 199 203 204 205 
207 211 216 217220 223 231 234 237 238 239 240 244 245 246 247 249 251 
256257261 263 268 269 270 271] = 134 

print (id ratings) 

[3433415255545144341352123325454554554 
5244344435455243224515535345251443513 
3244325343455255555535445445554453535 
3351424432251444411555 2] =134 
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(jpgsb uujssrrj 134 /_//_/£/<£( &gd(g rating QanQggjmmnrr. gtgstGgu (Lpgeb uuj<ssi(gdan<ssT 
ULLipiusbai^gLCi GLOfoaemi^Gu n^j amsmuuQLC). QuinggLona 2 _<sh<sn 1682 
ui—rhiaeiflsb 134 ui—idaifggdGa rating Glan^ggisnennij. (oTssrQsu rating ^eifldang ui_rda^gd(g 
0 STSS)ILD LD§)IUiShoSXS&T fBfTLD 6ULpfhl<9i G(5U<S5$r® LC). (oTSSrGsU <S4 Qgggna 0 -go LDlIQlD QufBgU 
<sfil<snrh](& ) il) ratings <oT<ss)iLb Quiurr Qansmi— Id array S-(gGundauu(^)d)fDg]. £g)g/ iSlssrsuq^LDrrgu. 

ratings = np.zeros(1682) 

print (ratings) 

[ 0 . 0 . 0 . ... 0 . 0 . O.J 

(gfigl 1682 columns-eb ^igs)ld gg O’s gou QufdnS}(gd(gLCi. iSlssrstsrrj gt gg ui_riia(Ggd(g ldlIQlc) 
rating GULpihiauuLLQmmGgn, ^Sdgg ^i—g^lsb ldlIQld 0 -go rgd£l GVLprhiauuL-L- rating- ^sb 
LDfTfDfDLd Q&iuu-ILD. ^jg/oana ratings[id movies -1] = id ratings Grssrd QanQdauuLLQGhisrTgi. 
^jgg ^i—gjgleb idmovies <srstsrd Qan®danLD<sb idmovies - 1 gtgst Qan<g)dauuL[Lp(guuGts)gd 
aGussfldasyil). sj-Glssnssfleb 2 sug] ui_gg]d(g 3 gtgs)ilci id^]ul 9 (^ ^<s(rldauuL[Lp(grBgn( 5 b, )gg 

array-eb ^p/smi^nsugna ^ovtLDg^lqgdtgLb 0 -go L£>njD^j 6 ugfD(g u&>l<sona, ^psirgmsugna 
sS86$)LDr5§))(i])d(&jL£) 0 -go L£>rr(D$®S)®Lb. sjQssresflsb ui^rd a Gift sir index l-aSlqgrBgj ^pidiSlddlgg]. 
^ ssrrreb array- esr index 0 -GSl(gggl ^rjLcnSlddljDgj. (gfigGnGSTg gGfilijuugfDanaGGiJ id movies -1 
gtgst QanQdauuLLQGfTGYTg]. 


^Gg (Lp&DfouSlsb G) LonggLCi 943 uuJGSTijaqsrijdtg 943 Id array s_(gGundauu(^Slpg]. ^gs)gu 
new data gtgs)ilc> Qufiiu LnliguJGVidfgGh syGijQ go nssr/n nad GarjdauufgldlGsrrDGsr. ^juGungi 
print(new_data[OJ) GTGsrd QanQggju unrrggnGV, (LpgGO uiussrrr rating ^GtflggjGfTGYT 2,3,4,5,7 
^t,dliu ^ji_fh]<ansifl<sb ldlLQlL ^ssxsua(Qnjdarrssr rating QGiJGifluuQGiJGngd ^rrsmserrid. 1,6 
LDjDtgjLD use rating ^iGifldang ^ji^rhiaerrlGb 0 GlGUGifluu(j))GiJG$)gd anemscnih. (gfigi 
iSUsilGlKnyLOnfQ]. 

[0.0, 3.0, 4.0, 3.0, 3.0, 0.0, 4.0, 1.0, 5.0, 0.0, 2.0, 0.0, 5.0, .... 0.0, 0.0] 


3. GLDjna<smL- (Lp&DfDuShsb ^ssxsstggi uudfddlg grjGyaG$)GfnL\Lb LDnrr)r& ^Gsnngg] iShssnssnf 
PyTorch Gjrpfrpd Qamsbmd a^ipiu array gol^g^go Long fry GugrpanGsr rfirrsb iShssrsiJQTjLDnfgj. 

trainingset = convert(training_set) 
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test set = convert(test set) 


trainingset = torch.FloatTensor(trainingset) 
testset = torch.FloatTensor(test_set) 


4. $!§] binary classification-da nssi problem (oTSsru^rrsb ^rrsyash ^smstsr^stD^iLjLC) O's & Fs <suL^sftso 
LDrrfDfDuGurrflGforrLb. 1,2 OTgo/fi) 0ss>/ d/5<5 LD&luiJCfiam O-^svild. 3,4,5 (oTs^iild ^errsflsb 

<S4G$)LDr5£ LD&tlULffl&m I- rM,<5VILD l_LDrTIT)fDL£> Q&IU UJUU ($)&>) GST (D<5ST. SJfD ©<£ SSlQ <5U ^/TSl ]fl<S$)& 

^sifldauui—rrgistnpd (^jf)d(^Lb 0 lc^Iuli-1 ^,<sb ^I—Ldit/b/diI) G)aiuiuuu(f)d}fD§]. {^iproamssT 
rffrjsb iSlsirsiKnyLcrTfQ]. 

training_set[training_set == 0] = -1 

training_set[training_set == 1] = 0 

training_set[training_set == 2] = 0 

training_set[training_set >= 3] = 1 

test_set[test_set == 0] = -1 

test set [test set == 1] = 0 

test_set[testset == 2] = 0 

test set [test set >= 3] = 1 


5. ^uGurTg] uu5)fD&)d(3jd QaGVidauuChUl) grrsyatsh iSlssrsaq^ii) suL^sfleb ^stsuDrs^q^di^Li). 
$l$ 5 lso 943 rows ldjd^jlci 1682 columns afTGmuufnl). 

print (training set) 

print (training_set.size()) 

tensor([[-l., 1., 1., -L, -1., -1.], 
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[-1., -1., - 7 ., ..., - 7 ., - 7 ., - 7 . 7 , 

[ L, -L, - 7 ., 

7 - 7 ., - 7 „ - 7 ., ..., - 7 „ - 7 ., - 7 . 7 , 

7 - 7 ., 7 ., - 7 ., ..., - 7 ., - 7 ., - 7.777 
torch.Size([943, 1682]) 

iSIgstsst] for loop ^LpsoiD 10 &ipiD@)aefr 2 _ 06 iv ndauu lL®, spsiiQ surras &TipfDdlufl^jiD 
uu5)jn@)d(&ju uiu<ssru(i)it5§)uj <s86fT<sy(!TjdaiGS)6fT Q&iliu-Ild rfl&Lpsii rBGsu—Qurgjdlingi. Gidgoiid 
lBgsstQid spqijfor loop ^Lpsoif nj!j(Q] jTjfTjDrra ^gs>id 10 Qgmgtslasiflcsb prjsyaissxsnu iflfl^gi 
uu5)jd&) ^siflddlio^]. weights, bias ^dhuesxsu uuSlfodld^u uiuGsruQgguuih— ^msiprFjd&sh 
^, 0 /i). ^jesxsu random-r^jjSsr lci^Iuli suss)erru Quiptgj ^juuuffodlGsnuf Q^m^fhi^dlfo^]. 
Gidgviid spshGisurrq^ driD/ndhifspiiD ^>f msif cj^daiGSim GiDiDutfif^isu^fd^ train]) 

gtgshid function-gou uiUGsruQgajjdlfD&i]. Input idjdqjid hidden Gsviufj&m £g) pern®id spGij weights 
id^IuiSIgsigstu uiusiru(f)f§]dssr/Dssr. ^esmeb ^gjoaimssT bias iDSduLfaGrr idlLQid 
G< saf^ju(fidl<siTfDssr. a gtgstu^j hidden Gseujq^ds.mssT^i. b GTSsrug] input GsoiuqTjdarTGSTg]. weights 
6T<siru3j] random-^s,^ G^rjrBQ^(f)d<9iuULdL_ (200,1682) gul^g^go <3fGS)iDr5^ iD&ylULf&ssiGfru 

QujDQ]^ §l<51J[hJ(3)LC>. 

hidden() ld fofry ld visible() functions ^jpsm(^iD spshQsurrq^ (ipGSipiLfiD hidden node-d&TTGST 
LorrS)filastsxsmLiil>, visible node-d&mssr iDTr^lfl^GsiGniLfiD G^rjrBQ^(f)ddssrrDssr. (g^GS) Gnash 
(f]rrsm(f )id 2 iD^luLfassisn^ (fqijuiSl ^GiflddlsiriDGST. (Lp^eomsu^] id^Iu l / spdjQsamr^ 
LDa^/flu-iiD sample-^a ^ssuDGijpfB&mssT flafp^assisud (^(Sld^ih. ^rjGssri—rrsiJgj ld§)uli 
B ernoulli (Lpssipuflsb G^fifQ^(fidauuifi_sample-^u QufDflfp^d^LD. Gidg$jid hidden () 

G)<suGiftuu($)£§]£)sirpGsr prjGifasiflsir sui^su id torch.Size([l00, 200]) gtgst <gfi(njuU6S)£iLfiD, visible () 
Q sijsifluu ($)£§] dl sir jo 6sr prjsif a <si?l gst <suLSf.su id torch.Size([100, 1682]) gtgst ^(njuussipiLfiD 
an<om<svnLD. 


print (phO) 
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tensor([[9.9967e-01, 2.0279e-21, I.OOOOe+OO, ..., 3.9762e-26, I.OOOOe+OO, 1.0000e+00], 
[1.1460e-09, 2.4450e-22, l.OOOOe+OO, ..., 3.7041e-22, l.OOOOe+OO, 2.0576e-08], 
[1.9920e-03, 1.8334e-ll, l.OOOOe+OO, ..., 3.2599e-21, l.OOOOe+OO, 1.1196e-17], 


[3.3063e-03, 4.9096e-21, l.OOOOe+OO, ..., 1.1215e-25, l.OOOOe+OO, 2.0377e-05], 
[9.8176e-01, 5.9265e-14, l.OOOOe+OO, ..., 6.0945e-25, l.OOOOe+OO, 2.3354e-03], 
[2.9210e-09, 1.6359e-ll, l.OOOOe+OO, ..., 2.2869e-15, l.OOOOe+OO, 2.2399e-13]]) 


^]LC>(Lptss)fDuSlQscQuj RBM, features-go£ CS^/rsi/ Q&uj&lfQgj. uu51jd@ (LpL^f^iSlssr, 

<$/76i/<£63>6)TL7 uuj<s$Tu(ff&ll hidden r£lgj,rrrrsirsi<ss)rnf §jjsssr(f)<siJ^ssT QpsoLD ^pssxssr rsmb 
Q^rrshmsomh. 
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17 Autoencoders 


Autoencoder sresrp rreb ppsi/^esierTp prrssTrr&Gsu sjGprr qqt, (Lpempuflsb (&jn$hi 5 hl® &(tjjd§i) 
<3jsmi)dad &pp]d Q^rrsh^dlesrp «£0 aSlstyiuif srssrpj Qurrq^sh. toTssiGsupassr dimensionality 
reduction, feature representation Gurresrp (ffji—di&siflsv £g)g/ G)u(ijifu[hi( 3 j sudlddlpgj. Machine 
learning-d&rnssT ^(SlfLp&d app<s 61 <sb PCA-®ou upffu urrfpGpmf ^sbscsurr, <31 Gp 
Gsussisvssnud Q&ufsuppgirT&i flu^psc QrBif-QsurrijSleb uiusfru(]fppuuif)if s^0 qSIs^ujGld 
autoencoder ^(Gjif. PCA <oT<s$tlj§i GpfG&rrff (LpempuSleb ^emmiLjif prrsy&siflstiTdimension-g>d 
(ajempdau uiusiruffdlpQpsSsb, Autoencoder srssTUg] GpfGs,mfLpsb <^es)iDUJ[Tp(non-linear) 
&>rr<oL] <s> <ss)sirLLjid ssisnurrshdlpp]. £g)g/6i//i> unsupervised learning sus$)&s$)ujd &rrfd,pp}. sjQssresfleb 
if(Lp <o5 )pufl svprrssr (gnShifl i_L(jf) ^s^Lcds, Gsusm(f)if srsirupp(p srrfps^rp uufpdutjif, 
GULflarrLLQfpsgiif ^gstpSi £g)g/ aippjdGlsirTshdlppi. 


£g)<$ 65 T (Lp 3 )<S 0 rT(oU 3 fl GsOUjf ^_shs^lf(f] GsOlU if STSSTpj <3l<S$)ipd&UU(i)\lf. < 3 l®pp®p§! 2-6h6fT&jl 
encoding i£>ppjif decoding Gsmurf&Gn sjssrpj <3iss)y)ds,uu(fif. (gfipssr GsussisoGuj 2_msrtifif)p 
prrsy&svsrT syGprr <s £0 (LpempuSleb (ajrfiluHli-L® frfWjdld) ^sifluupj mppjif ^d^fflufifuj-smssr 
fidaif G)&iu§] i£sm(f)if uesiLpiu r/flssxsouSlsb ppsiiasmen Qsusifluu(f)pp]<sijpi ^(gjif. 
sussu—dhurrs; ^shsnpi (ohsuGifhifif® Gsmuf. Qsueifluu(frppjdlsirp iDSfluL/if, ^shsrrifif) 
Gsmuf surflGuj Q^svidpuufkidlsfrp i£>§>luL\if s^ssrprra.Gsu <3]&nLDiLiif. ^jssu^ufleb encoding 
mppj if decoding Q&iluuu uiussruQfdlssrp 6£0 s^s^ujGld 'bottleneck' sresrpjif 
<3j6S)Lpdauu(if)d)fDaj]. £g)g/ Qurrgjsurr&u ui—ihiasiflssr ifig] Q&ujsculL® GpssxsuuSlebsorTp 
$li_rhigi6S)GrT ^aropjGiJSfi (denoising image) , urflLDrrsmdi&.ssrGfrd (ajstnpuugi (dimensionality 
reduction), «0l/l/ Qsuefrssiemj ui_rhia;GS)GrT susmsmu ui—rhi&srma umppjeugj (Image 
colouring) , ui^pdfsir (Lpddhu tSrjif&di&smsfT ld if (ft if GprfpQpQrppi GhsiJ 6 i?luu(if)pg]< 5 ijaj] 
(feature variation) , disc UL-ihiasiflssr ifipj (ffrusburrs, Lord Shu rghzmemiHleb <3]d(fi_uuif(if)p 
Qpssruffidlsirp s,ss)i_ufssr Qumf, uSfluLjiflssnD Gurrsfrp 6 T(Lgpajid&iG$)GfT fid^sugi (watermark 
removal) Gurrsirp s&siyiudi&smsnd Q&iusupprp) (ff§l Qurflpjif uiusiruQfdlpgi. 
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Demising Autoencoder eresrug] s ^0 ^L^uuesru. si/sai*. £g)g/ desr^f^ rflesreouSleb 

$l(njd(gjLt> ^rjeijaesrenGiurr/uufssr^Qiurr erd^§id Q&rressrd <3psiJjDj!5l<sSl([^d(^id 

<suegjeurrssT <31id&ihiaes)err (robust features) ldlLQld ssreugg] idemdid 3 [f^ > psrj 3 : ,ss)mQujrr, 

uufssr^Qiurr urirp^Qrjsurrd&id Q&iuuju uiuesrud^jD§l- 

Sparse Autoencoder eresrugi ^jesruuuilu Qeoiufleb ^eherr nodes-esr srsssrsssfldssr&ssHud 
( 3 jss)fDdairTLC>< 5 b, eh erf if d iDjo^jid Qeuerfluffd Qeviuflev 2 _eherr nodes- esr 

<oTsssressfldes)ad(3)d &L£>uma(l>GiJ ^ssuxiddlfD&ii. tffshsurrrQi (ol&iLnLjidQurrgji grjsijaesrerr 
encode Q&iuiuiTLCKsb epehQeurrqij node-id 3 [uurpQuj rflssrssrsfilsb ejjDjSld Qsirreh^id 
<31 u mu id ejrdudid. £g)<$s 3 >s 3 r<$ ^silrruu^rQ&rr&i epehQeurrq^ Qeoujiflepiid spqrjdeo nodes- 
esr Q^iuevurrfssru (Lpi_dd) ssreuuu^ssr ^tpeoid Information bottleneck-go Gl&iuebui _ 
esreuddljogfi. 

Deep Autoencoder eresru^j (Lpgeomd rflesxso, ^jpessrumd rflesreo, (gpesrjDmd rflesxso eresr 
uebQeunpj rflesreoaeifleb 31 d^®^^ features-god ( 3 ) 6 S)jD£&jd GlarressrGi— surBgi 
assrudufsb < 3 ] suit) esr rod (ajjShfilfd Gsreuddl/Dgl. iSlesresrp^id^jSliiffL^esresr^daiid 
Q&iiig], < 3 if^peii< 3 i 6 S)efT QsueUdQ&rrsmd eucj^eij^id £g)G>,$ (Lpesrjnufleb uevQeufru 
rflesxsoaerfleb [Besu—QuitydJjDgl. Image Search, Data compression, Topic Modeling, Information 
retrieval GurresrjD etfefyiurzi&erflsv Glu(ij)idu[ij( 3 j eudlddlfDg]. 

Contractive autoencoder sresrugji Quujfli—uuurr^ apjeyaes)err (unlabeled data) S £0 
( ^jSluiSlfi _ label-esr dLp &(njdd) Qeueifluudd,&> ^-^eijdljD^]. ^j^iQurrssr^j 
^_ 0 Gi//T<s«UL/(J)/i) QeoiSleb latent variables erssrfjjj 3 ]SS)Lpd< 3 iuuddlssTfDSST. 

Regularized autoencoder-sir sp(rrj si ]SS)& ^(gjid. 

Variational autoencoder erssru&ji spQp uu^^sc dtgjdigj LDnrdrDrhi&ssisrrd Q&iiigi L/^/uqg; 
ui—fhi&ssrm ^_ 0 Si//T<a 0 si/< 5 /D 0 a_^siy®/D^/. [bit id ejfdQsissrQeLi ^gjiQurrssri d 
s,rrQsssrrrerr]s,ss)erru urr/j^^lQ^uQumd 3 t<sbeo 6 urr! (Lpgedleb ep(njSii([ij6S)uuj (Lp^^leb 
^rndiSlfgj < 3 i^leSl(r^[ 5 §] d^jd^j LDrrrdiDii^esrmd Gl&iug], uecq^ssruiu (ipariiaeh 
Q^esrud^LDl £g) esieuGhuebeomd Variational autoencoder-esr QeuesieoGiu. Graphical models 
z>_(fT)S\j nd&id LDfogpid image generation QurrssrjD ^i—rhi&efileb Qu(rrjidurhi(&j 
eudlddlfDg]. 
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18 Reinforcement Learning 


asssflssfldGarr ^sbsvgj asssflssfliurrsb s^q^su rrdau u lLu aq^sfldGarr s ^0 sflsyujpsnp 
SilQTjLCiug §)l(ijjL£>u& Q&rrsbsSld Qarr(f) u u^sir Qpsoif ^^ss)sstu uuSljD^jsrflda (rpiusonf 
(Lp<ss)fDd(&j 'Reinforcement Learning' stsst^j Quiurj- aiuuma spuda^L^UJ LodlLp senrj$Bl(self-driving 
cars), asssflssflGiurr® wdas&srr aSissxsmum^d QaiunLfif gaming industry Gumdr/Dsurofflsv syq^ 

&([ljaS}d(3j{5 StlfOLDUL _ Ullf/ndl ^Glflda £g)<95<$63><SUJ (LfiSS)ID uujssru(fd,^uu(fdl(D§i. aifuf^lsb 

S— 0 sorrdauuifi _ 'Deepmind' srssrug] 6£0 £)/d/ 5<5 S-^rrrrsmif ^(gif. ^ggsvaiu 

Reinforcement Learning srsu su rrnpj rsss)i—Quay d)jog] srssrug] ujorfil £§) uu(&,§)!uSlso srSlsndaiimad 
an<5m<svnLD. 


Qurrgjsurra spqjjdso stflstyiurdasnsn (3jLprBss)pd(3j rsmf <sp(r^(Lpss)/D Qarrsv<sSldQarr(f)f^nsv 
Gurr^if. < 3 ]§] asouumad arorryd Qarrsm(f) sSlQif. ^j^stsrssr arr^rnjsmLDrra learning STSsr/rg 
^g>/ smipdasomf. ^ssrrrso Lifilfgi Qarrsnsu^rrx^d arorrgd ai^ssrumssT sfilsiyiurhiasinsrT Zfiqijifud, 
§i)(TijLDud QarrsbsSldQarT(f)uu^sir QpsoGw spqij (^Lpfss)^d(^ pmE l jfiliu smsuda (LpeyiLjif. 
^Gusurrirg tglrrrjifup $gl(njiEud Q&rrsbsSld Qarr(f)d(^if (rpiufdluflsb, spsuQsurrqTj (Lpsisronjif 
(3)Lpr56S)£ufls$T L//f?< 56 y/a 50 sjjDjDGu ith)] rsmf <S>1 ($)£&($) £§] QarrsbsSldQarrdda (LpiusbsuGa, 
'ReinforcementLearning' srssruu(fif. sr(f)^§idamfurTa S £0 (&jLprfsrs){Bd(&j ai^arrrj (Lpshss)snu 
urrrjpgj ldsssU asm(f)iShjyda anjrrgd QarrQd^LD Q&iusnso reinforcement learning-so snsug&tfu 
QurT(rfjf§>]u urrijuGumf. ^j^lsb uiusiru(ff > a,uu(fif (Lpddiu su rrrjgsngasnrTssr Agent, 
Environment, State, Action, Reward, Penalty, Policy ^diusufstn/B ^jdQaiusflsb ssrsuf&gu 
Ql//t 0 ^^)ly urrrjpg] rBrnf nrflrBag Qarrshsn (LpiusbGsumf. 


1. (Lpgsfilsb s^sst^jGld Qgrfhurrwsb ^rjifiSlfng ass)uduSlsb au^arrrj (Lpshssisrru urnjfgg Ldsssfl 
Q&rrsbso arDQ]dQarTsh(^if (ajLprBsngprrssr ^f\rhi(&, 'Agent' stsstjd Quiuflsb ^ss)Lpdauu(^iSlrD§i. 
<S>l£rTsu§i <oTgjD(3j rBmf arnrgjdQarrQda (LpiusbdGrorrGwrr ^sg^]Gsu 'Agent' ^,0 if. 


2. loTsnpu u/f/fld ajDQ]dQan(fida (LpiusbdGprrGLDrr ^^]Gsu 'Environment' stsstjd 

Quiuflsv ^ss>Lpdauu(f)d/D§]. aL^arrrnf, ^§2 so s_shsrr 60 dssrssrrf^drSluj GarrQfasrr, (ipiLash 
s-srrsrr ^Issia, s Vg/ (^rfilddlssrro srsssrash ^diuss)suGuj £g)/i /0 environment ^(gjif. 
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3. Environment-d^sh <oT[B^ rffesxsouSleSlQ^rB^i rsLogj arf)iSl£t5<ss)<so£ Glgrri—rhiig&lGfDrTGijm tSy&jGsu 
'state' stgstiqj <s>IGS)Lpdauu®LC>. aLq-3irT[j£$5l6b (LpiLgush ^(nj&tgjih gfDGumsnpuj $<s$xsod(& ) 'state' 
ioTsstiqi Quiurj. ^enguLirTijggi LDSssflssnud asmddhl® Q&rrsbsviLb Q&uj<svid(&, 'action' toTssrigj 
Quiurj. <S4 &> it <su &tf environment-<sb 2_<sfT<sfT state-d(3j <sjfDfD<S)JfTfru, agent aargiCTLiu action-go 
GhsuGifluuQpajjLC). 


4. Agent Qgijgi?Iuu®£$uj action &jliuns, ^ssiLDU-iLDULl&g^sb reward-id, ^suprrsi 
<s$) ldu-IldulI&p^< sbpenalty-ih dUs$)ud(^ih. ^i^rrsu^i 'gendau u ®£§]g<sb' ldjdqjld 
'^ 6mU6$)6$TlU<Slfl^<sb' 576$) ILD Q U rT(l]j 61?l <50 £g) 6$)6U £g)/5V0 UUJ6$TU(]jd > 3 > lju($ldl6$T(D6$T. 3U7f_3irTp 
(Lpshesxsrrg djrnyUiSld, ^Itr^uiSl <sp6uQ6un(Jij (LpemjD (& ) Lpr56$)^6$)UJ L06$$fl G^u^idGungjid 

■pjliurrad Q&rr<s$r<ssrrT<sb spqjj i£hdurr6$)ujd Q&njjggi mnd&LDSirluug], gsurorrad Q&rr6$T6$rrrsv 

< 3 i^ 6 $fll_LD ^_< 5 fT< 5 fT l£\L -1 _ niLISHSlflsb 6 g> 6 $T 6 $)P[)Ll lS)(])lfhl§>ld Q<S/7<S)TS1/^/ GurT 6 $TJD Q&Lusv&Gm 

£g)/570 'reward' r$!so< 503)1 'penalty' 576 $rrru ^ 6 $>Lpd< 9 ;uu(j)dlfB§]. LShlumu Slsstu^^rrsb ff,rr6$T 
&jDf!)jd(olarT6mi_aii] &jl <6T<dr^]iL, l£1l1i_[T6$)uj (^Lprs^rrsb prrG&r ajDfi)jdG)airT6mi_&i] ^su^j 
5 T 6 $rr^jLb (&)Lpr56$)^ Lj jlr5g](ol&msh(<sri)L£). <0T<5S1G<5U (Lpstnjn&isiflsb 

LSlLh—muassKsrTu Quqjld ^dsuddlsviLD. sjfdQ&ssrGsu QufDvpdQ&rrsmu LEhdumu&smsfT 
®dPp3)I sdud^urr^j 5T6$TfD uiuddlsond ^ssrd^d dl6$)ud,ff > ^ 316 $)!usudss)#, 6$)6ud,g > i efliurra 
L7)6$$fl G)&rT<50<50 (LpUJfQ did(3)11). 


5 . & jl UJ fT (SST LD6$$Hs$nud asmddll— (&jLpr5<S$)£ gj 3 jJ 6 U 6 $)IJ UsbGsUgp 6 ULf\ 6 U 65 ) 3 i 3 i 65 )<Snd 
&6$$r(i)iiSliq-£3)i 6$)6u£$5l(rjjd(g)LC>. GL^arrijjljlsb s^srrsn qguQgu rrqTj dl6$T6$T{d^dfjuj 
GsinQ\ 3 i 6 $)<sniLjLh &6$$rd£) u(j)d a^^jsu^j spqij <su<ss)&. 5 -ih sumuuumlesu—u uuJ6$ru(j)jjld 
a6$$rdd)®6u§i &fD!QJ Ggjjg suena. gjgiGurrsirfgj <sp(i]j aSlstyiujsm^ rsi—jpu uiussTujjjgtfih 
usbGsurgj 6ULf]<5U6$)3i3iG<sn 'strategies' <oT6$rro Quiujsb ^ssiLpdaujul^ldlssrpesr. ^l^l<sb dlfors^ 
optimized strategy-goj Ggrjsy Q&iug] Agent grresT rjaLpjp Gsuemi^iu Q&ujgsxso rjlaLpjgiLD. 
gjjp6$)auj utsbGsu^j strategies-go shsrruddl ujG^ 'Policy' ^(3) id. 
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^gjlGumssrjD 'Reinforcement Learning' gtgstu^] supervised ldjd^jld Unsupervised GTGsrro 
$]!J6m(S)d(3)Lt> ^ sts) ld nj if. <$jQ Gsrssfl go (Lpgsflsb unsupervised (Lpstn/Duflsb 

iSUsstssrij agent-sst Q&ujgsxsou Qunfgjpgi <= gy<$<s3><s37<$ Qgrri—ijddluJiTa 5<gj SOqrjjB&tfGu^Gsr QpsoLb 
supervised (LpoDfoufsb Glgrri—pdlipaji. aisssflssfliLji^ssr pmE sflov)smurnf) dlsirro dUifidaiLLif) 
(sfihssxsmuml®,, Chess sflstsxsmumftf), < 95/777 urs^uJLD GurrssrrDsu/D/flsv r 5 L£>(Lp 6 S)i—UJ gt^I pessfliurra 
$l(njd( 0 jLt> aGStifl<ssf)d(3)U uuSIjd^J Gaidai reinforcement $£gji<surh]&G<SfT 

uiu<oSTu(f)dl gst ff) gst. (Lfigsflsb asssflesflujmssTgji Qg/flrsp Gustnaifilsb pldGldtt® 

<sfl&sxsmurTi_f Q<s/ tl_ [hidlGsrrTGpi ld, iSlssrssrij r 5 Lb(LpG$u_uj rBi—Guui.dGS)aiai(^d(^ gjtdtdttij Guttgst^j, 
4>/<gjg)/«ni_m stfhssxsmurTLLQu Gundena 5 LDnrdrfiJd Glaimsm® Qsu/D/fl Qujd (Lpiusviif. gtgstGgu 
<3i<s$)<ssTd > dj(D(& ) Lf Qurr(Tijr5&i]L£> susm&uflsv ^j^^^surhisnsfilesr frQTjdarziaiGsxsmj upGUGorrau 
iShsinsucjTjLDniTp ^G$)LDda;Gomi>. 


( ■9jLprfj)<s$xsv<s$)Ujd &<sm&n&s$fld,& ) 6V (Observation of Environment) 

(^LprflesxsouSleb s^gttgtt ^/dGu[tgs)^uj rfhssxsod 0 <oTrhir)Gumr)i Agent &>g$tgs)IG$) 1 —UJ ^uddp, 
G)&iU6S)6od> Q&,rTL^pd>&,6V (Act according to the current state) 

fr,sir65)1 iSD i _ uj (ol&iucsydamssT gmda ld^ul/ ^gogo^j g6mi_<ss)<ssr£ Qgmstnaistnuju 

QujDggd Q&,rr<5h(@(T)ff><sb (Getting reward or penalty) 

Agent ^mssr QujDgjdGlaimsmL- ld^IulS}jd(^ gj/d/DGutt^j ^asrgi/flnLiu Q&iuGsxsoiLjLh, 

Lj fl^GSxsou-j ld LDrr/D/fld (olaimshi^^sb (Finding a strategy from experiences) 

Agent rfl&Lpp&pLCi action gtgstu^i (^LprflesxsoeiDUJ Gpqjj f]<ss)<soufl<sS\Q^f§i r£lGS)God(3j 

LDiTfDfQj if. g$)uGurT§i (SjLp f) <ss) so uf! go sjipu ififimm LDnjnjB f}<s$)<sod(&j gjtdtdgutt^] agent 
GLDjnansmL- Q&ujGO&GsiGfT i£<sm(f)Lh LS<sm(f)Lh Q&iug] u<sbG<sufrg ^Lp/odlanssnsn 
S-fnyGurrd^iE. ^Lpfodlansiflesr (Lpi^G^GO, <3i§i Qujf)Q]<sh<sn reward ^gogo^j penalty 
L£>$5lUGS)UU QurTgjgaj] &ifl uj itgist gu L^}(LpGS)fDGS)Ujd &Gskr(d iSlip-d^if. (Iterate until finding a 
best strategy) 
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(o)d5/rz_<95<£ $65)60 3Lp$d)36iflsc Agent-0\$3i—.rjdd\uJ33 2M33 ld$Iul/ 3G63 
®650—d(^L3)ULi3$$6b 3 ( 7 ) 131 33u l/$$§] 0l336S57®6i5l lLl-,3)] 67657 $6S)657$§i 3Lp$d)365>63 

§16033$$G6oGuj $^j$$ 6)5h—d3^i—3§i. (Don’t be greedy). <31$& u lL 3 3Lp$dl365>63 $3if$$ 

3/$60 G^37J3UJL3)33 ^773313)$ U L/ 3 63 ®65)l _ $3)]6363753 3160603,] <^65571—65)6S7$03,365)3 

®650—$3^6363^3 67657U65)£U Ou37g$G^ 3$UJ3657 $[TL3)3657$65)75 G133)Q336363 Govern® 13. 
^j$3,$ $$L3)365113365T3J OeOTgLC) 6y(3) $65)6065013LJ Ou37tf$3tf 3165) 13) UJ 313) 60, UGoGgO^J 
3 3 go tzi 3 6ifl go <3)65)13 $3, ugoGgo^j $65)6036ifl65r Ol£3i—rjddliu 33 <3j 65) 33/13. (sequence of states 
according to time) 


<31®$&&r73 6p([^ sm$$65)UJ 3 ,65765fld65>3UJ33 ^jUJfh]3 65)6033 reinforcement learning QpGOih 

6760603(Q] uufrodl <3/6lflUU§] 67657U65)g d if3365571— 67®$§1333lOl7/.6b 33655760313). ^j$^65)3UJ 

UuSlfdlufeir Gu33)1 U UJ 65T $3 65)63 67[d®(3j$3)l GJJOJO G606557®I3), 6731(3, <ffrQ33 G606S57®U), 

^]65)L _ LJUlLi _ 3ft!J$65)3, (3)65)3)3,3, <316361) Gr 5 U $$60 3lflUJ3657 $65)3 Gt,33® 67 6060 37g 31—3,3)1 

Q360603)] Gu 36573) UGoGgO^J 6)5) 6fy 3130365)60 75313) 37)7g$3,7JU Gu 3 ®Gj 03 l 3 ). 6£(3j U 3165765) IJ 
3fllLI3657 (sfjl—$$63> Q33653® G3$3(3)13) 0)31316013(3 <31$3) <316367/ 0)60(3) 13)$365)637)13), 

6 )>6VOl6V3(IT) (Lp65)fQ 75603)3657 $65)3 GrB33®$ $(3jl3)L/l3)Gu03)lL3) 3jDQ]d (3j65)f0$& 3,65571—65)657 
13) $ LI L/365)63 <31677)3,3)13, $(3)$§]60g657 Qp60L3> 2enrf$3(3j3 37077)]$ £7J (LpiftlL/lf. $]&JD(3)$ $(&>$& 
L3) 3$ ft UJ3657 3,7367/365)63 Gym 6765)113) module -3(3)63 33655760313). (sfj^65)657U LI 3165710®$$ 

G 13 ) 7 ) 365571 _ 6 )$ 6 fy 3100365)60 0 ) 33160 ^ 7)33657 $ 36003 ) < 3 /^ 7)33657 6 ) 516333 (Lp 13 ) if) 65760 ( 3 , 13 ) 37 }]. 

https://gist, sithub. com/nithvadurai8 7/d25a4bdf226a 7b5df92267e23ce8d28e 


import gym 
import numpy as np 
import random 

e = gym.make("Taxi-v 3 ").env 
e. render() 

e. reset() 
e.render() 

print (e.action_space) 
print (e.observation_space) 

e.s = e.encode( 3 , 1 , 2 , 0 ) # (taxi row, taxi column, passenger index, destination 

. - . -- - 
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e. render() 

print (e.s) 
print (e.P[328]) 

epochs, penalties, reward = 0, 0, 0 
while not reward == 20: 

action = e.action_space.sample() 

state, reward, done, info = e.step(action) # (284, -1, False, {'prob': 1.0}) 
if reward == -10: 

penalties += 1 
epochs += 1 

print("Timesteps taken:",epochs) 
print("Penalties incurred:", penalties) 

q_table = np.zeros([e.observation_space.n, e.action_space.n]) 
print (q_table[328]) 
total_epochs, total_penalties = 0, 0 
for i in range(l, 100): 
state = e.reset() 

epochs, penalties, reward, = 0, 0, 0 
done = False 
while not done: 

if random.uniform(0, 1) < 0.1: 

action = e.action_space.sample() # Explore action space 
else: 

action = np.argmax(q_table[state]) # Exploit learned values 
next_state, reward, done, info = e.step(action) 
old_value = q_table[state, action] 
next_max = np.max(q_table[next_state]) 

new_value = (1 - 0.1) * old_value + 0.1 * (reward + 0.6 * next_max) 
q_table[state, action] = new_value 
if reward == -10: 

penalties += 1 
state = next_state 
epochs += 1 

total_penalties += penalties 
total_epochs += epochs 

print (q_table[328]) 

print("Timesteps taken:", epochs) 

print("Penalties incurred:", penalties) 

print("Average timesteps per episode:",(total_epochs / 100)) 
print("Average penalties per episode",(total_penalties / 100)) 


rfiu 6pid&rT6ST aSlendaiL u>(6truth QouefiifiCfi: 
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1. gym.make("Taxi-v3").env eresrd GlarrQuupesr ^ tpsoid uuSIjdHf) <s4ei?luugfD(3j£ $(&>&$ <m,Lf> riles) so 
2_06iv/T<s«uuL(5)6i5)(pzA ^pesTLSg] Q&iusbu ® ld render() ld/dqj id reset() ^Shusstsu 
(3i,Lpriles)soes)iu QsLisftld&md(jd&>sb LDjorQjid (g ,ipriles)soii5lsb mnrjSil ^errerr £§)/_<$ 

G^rrrrmuiDrrsi umroriil ^ssild^so Gurresrro Qsuesiso&ssxsrrd Q&ujiLjid. £g)g/ iSlesrsu(r^iDrTjr^j 
&rTsmuuQ)id. 



P Agent: £§)($)si) arressruLiQid &lr£luj LDQjnj&err rilrou Quidi^^rresr Agent, ^gy^/nsi igi ^^i^rresr 
g6sr6sfl<des)&ujrT& ^jiurdSl uiuesrrjs.esxsn ^surrsurj^ffluj Gl&esrgj G&rfda 

3ijDQ]d Q&rTshsfTLiGurT(3)ih 

P Environment: ^jd^Lprilesisoiurresr^i sreifliu 5*5 emr^eu ^essfl QurresrjD ^es)LDUL]i_esr 

S_(77j 61/<9>Z_D Ql&lLlILI LI LjQ) id. ST6 StQsU (g^^lSO 2_efTGfT SpshQsurTQrj &I6STSST SlSSTSST 

u(3jGilss)UJinLb rSHessfluilsb uiuesru(i^uu(i)id £§)/_ LL&>luLi&ss)efT ssieugsji ereiflgrra 
^esstiasomd. ^i^rreu^i R, G, Y, B ^ShuesxsoQuj uiuesrrj&esten sjfDn$l ^jrDd^eu^rD^rresr 
^)/_/5/<95ot. ^esxsu&eh (LpesiroGiu (0,0) , (0,4) , (4,0) , (4,3) eresr^j ^smi&uu Child. 
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> Episode : (gfjShsv (& ) Lpr£l 6 S) 60 <d(& ) sjffu Agent Gl&spid&ilid 'Action' STSsrug] usbGsu^j action- 
aetflGST Q&, /tl _pd dhurry ^ss)LHLjid. Agent-sir s^suQsunq^ Q&uj<sp]ih (^LprflssisouSlsb 

LDrrjDfD&sng sjjdu^^^ild. iSlssrosnj LDrrfdfDUUL^L _ spsiiQsurTQ^ (^LprfJssisvd^id spffu 

Agent ^ssr§] Q&iusnso rflaLppgiid. £g)<3.$ (LpanrouSlsv Qgrri_pr5gi Q&iusbuid® 
asmi—dluSlsb uiussrstnp ^suqTjd^fhu (gfi—^Shsv Q&sirg}j G&pd(3)id Q^iusvrrssr^] spqjj 
'episode' srssTirgj ^ssiLpdaujuf^dljD^]. 

^ Action Space : gefrpS) ud sir Gl&iusburrQfasrr iSlsirsup^id 6 Q&iusbasiflsb spsirforra ^siminLjid. 
2Mp$SS)lLI Q^fd(&), Sl/i_«0, &lLpd(&), GlDID (g ^t,dlUJ ^ISSIffSiSlflsC SylQTjLJLgSU3j} (LfiStofoGuJ 
0,1,2,3 srssrsyid, 2 mpS>is$)UJt 5 §isud(&jLb Q&lusvild. rSJfgjpgjiLb Q&iuspjid (ipsinpGuj 4, 5 
srssrspid (^rSlds,uuQ)Lb. ^idix>^luLj3iGsrT 'Action Space’ srsirrp ^sftLpd&uuQ&lsirrDSST. 
print (e.action space) stsstu&j 6 srstpud ld^IulSIssisst QsusifluuQfgajjsiJstnpd arrsmsorTid. 

^ State Space : (p^Lprflsinsvudlsv s^srrsrr spqij r£lss)sod(3j sjjDjDsurrGjD Agent Ksirsnissu—UJ 
Q&ujsmsod, Qjzm^rhifsjih srsirgp ^ffilGsumL. Qumppid srppstnssr rglstnsoasrr {gfjrs&d 
(^Lpril&DsvuSlsv ^ssnDiusvmd srsiruGg 'State Space’ srssruuQfid. spshQsurrqj) 
uiussrsfrpupd 2 _sh <s 4 s$)Lp£§id Q&>rrshsu§} spcj^ rfl&nsv. ^gGsij 5 uujs$rp&,if5fr ) d(3 ) 5 
rflstnsvasrr (R, G, Y, B ^dhu (gfii—rfjasiflsSlQTjrBgg 2 -srr ^ss)Lpd,g > id G)arrsh(<snjid uujssrp&srr 
paSlp spjdQ&sstGsij &rT(njd(3)Sh spqjj uiussrp ^(njuuprr&d asmddlsb Glarrsm® 5 uiussrp 
srsirdlGfomd). c gv(J)< 5 < 5 < 5 /T<s spshQsurrq^ uujssrssipu-iid <sdsupsu(rrjd(&j$\uj ^ju^^lsc 
Q&sirggj G&puugi r3i®^ rglsmso. 4 rfJtgj^p^fnarrssT (gfiurhi&iisrijid 4 rfissiso&srr ^(gjid. 
s,ss)udiurrs, 5*5 sm^sussiuoussiud Qarrsmu (^LprflssisouSlsb arrp Ss) 0 zi>L//i> spsdQsurrqij 
fdlssi&iLjid spsdQsorrq^ rflssisossiiud (&,pSld(&,id. srssrGsu Qum^id 25 dilsn&asiflsb &rrp 
dfqrjidu sumuuLi ^shsrr^rrsv, 25 rglssisoash 0 si//T 0 /i>. Gld(d^(£Iuj 5 uiussflasrr, 4 
rffnpj^rhiash, 25 ^ssi&asrr^dhu ^tss)ssT^ss)^iLjid G&rjggi state space 500 ( 5 * 4 * 25) srssr 
^ss)i£>iLjid. print (e.observation space) stsstu&h 500 srsstud ld^Iussiu 
QsusifluuQfg&psiJssipd &,rrsmsorrid. 


2. ^uGurrsg / 5 /tld/t<s s £0 rffssiscssnu ^cj^so rrddld Q&rrQgg], ^^ssnpuussiuudsb ^mp^lssnud 
sijD^jd Q&rTshsrrd Q&iuuju GuadlGjornd. e.s = e.encode(3, 1, 2, 0) srsirugj GlarrQfdaiuuili— 
(^LprflssisouSlsb ^mp^^lssnu (3,1) srssiiLSiuddlsb ss>sud(3)id. rSyQ&g] jBpuuLLdfshsn 2, 0 srssnid 
STsm&rn passenger index ld if) fry id destination index-god (3jjr5ld(3jid. e.renderQ srsiru&j (°f]r 5 &,u Ljtflm 
r£lss>svssnu idlssTsaq^LcrriQ] Qsusfrld&.rridQfid. 
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3. print (e.s) <oT<s$tu§i 328 (oTs^ild (oTsmssnsssr QsiJGifluu®£§iil>. ^^rrsu^i Q&rrQd&uuLh— 500 
rglesxso&ofihsb £g)g/ 328-su^j rghssxso lorsir^j s$ l[5& rffssxsod^fluj Qsu^LD^ld&nrssr 
<^LLi—GiJ6S)6m iSlssTsuqijLDrrfru ^smAiLjLbprint (e.P[328J). 

{0: [(1.0, 428, -1, False)], 

1: [(1.0, 228, -1, False)], 

2: [(1.0, 348, -1, False)], 

3: [(1.0, 328, -1, False)], 

4: [(1.0, 328, -10, False)], 

5: [(1.0, 328, -10, False)]} 

<SULpsu<SS)LDUq {actionl: [(probability, nextstate, reward, done)]} <srgn /ld ss)LDUi3<sb (s]i([jjd(3 ) il>. 

£g)Si)<a> S-shstT 0,1,2,3,4,5 ^Sliuemeu Agent rflaippgiLb Guetnanurresr Qa=uj<sbass)sntLiLb (pjtfildSilpgi. 

<91®P§1 2-Grretr 1.0 ctotu^/ Agent sydjQsnm^ Q&ujGsxseupi) rglaLpp^iGijpparTssrprobability-g>d (^n]ldd}p^j. 
^esxssrpgi Q^iuebai^ii) r56S)i_QupjGLjpparT6ST GurrtuuLi &iDiDrra(3GiJ ^j(T^d(^LDrTp<somsb ^pesr 
€Ti3(3urT§iLb 1 gtost ^omDupi). 2- Gnsn428, 228, 348...(3urTGsrp GTStssra sir Agent ^d(^r& u f_5? ll_ Q&ujgsxso 

rgJaipdtBlGSTrTGi) Q&6srp6S)i_iLiLn rflGsxsodamssr GTSssrsmssstd (&,f$ld(&,ih. gtgstQgu pmssr 3 GTGpnFli^dSusb 

<sjrdQa6sr(l>GiJ genrfSil 6S)GudauuLl($)erTGrTprT6piLb, 4, 5 ^dliuemsu uiuesresip <ppr& ^pd^euppamssr Q&iusbaGrT 
ldlIQGIld GTGSTuptr^jLD 328 files) <sou51<sb mappid <ppui_rTLD6b ^(3p GTGmrGsxsssr Qenedluu(fifi^HLieben§i. 
<£l®fi§l 2-Grretr - 1 , -10 ^ > dhues)en e^enGearr^ Q&iuepid(&)LDrTGST Qgu^ld^I ld^)ul/«iot *%(&)Lb. aes)i_d) 
Q&lUGpldatTGST QGU(3)lD5il LD^IULI LDl2($)LD, ^pGSXSSTUjQpgl 2_6f7W LD^IULf True <oT6ST LDtTpi LdQ UtT§l 20 GTGST 
^emDupD. ^prreugi a6S)i_d)u51<sb osben False eresru^i uiuesrestn (£[gstgsiild ^GUQTjdamssT (gfii—pSileb 
^jpdSlaSh—aShsbissxso eresruesipd (a^rfidSilpgi. ^eaes)p QeiLpfflaijLDtTa ^pd&leFlFi—FilGsr, £§)ldld£ 5 )ul/ 

True GTGsreijLb, a6S)i_d) Q&iuepidarresT Qgu^ld^I ld^Iuli 20 eresreiph umtfiefiUfiLb. ^giQsu <sp(jj 'Episode' 
fiempeypppeopd (^tfid^ih. 
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4. ^juQurrpi RL-damstst atpp «^© algorithm-GgU-jih iSlesruppmAGb, random- ^s, senrjSiletnuj ^tifipptfipp Silas)# 
Qptrddl parr asxsuuuppamssrloop iSIssreuQpLDrrpu ^snusu-iLb. ^jpi a gs)lJf\ Q&uj sorrssr drop-off d(pd 
dlassi^ddlastp QatpLcfl LofluL/ 20 arast ^assLciunp soasyrr ^entjfiasnu Qpm—tjppi destination -®» QptrdSlu 
uiusssflda ss)suddlsstro while not loop <^,© lc . e.action_space.sample() or air u pi Qumpp Qaiusbaetflsb £§)©//©/ 
random-^a 9 © Qaiuesxso arQppi rflatpppiii. e.step(action) srssrupi ^dQaiuesxso rflaipppiGijpmsb GippSlesrp 
^i®pp rfUesxsodarTGST idSsIuli, ^dQ&uJsgidtpu QupiSilasrp Qeu(piD^I ld^Iul/, ^aQatusomsb arrifhiJLD 
QpppiuQuppprr ^isbesxsoujrT (oTGsruesip (284, -1, False, {'prob1.0}) GTsirp euipesSJsb Qeuetfluu<S)ppiii). 

( Lppsi) Qpahpj LD^IuqaQefTstate, reward, done Qumssrp variables-<sb Q&LfidauuQSilGSTpGST. <spsijQsurT(p (jpasyp 
peuprrau utuesremp ^jpdSlaS}(f)ii)(3urTpiii), penalty loSsIulii—Gst 1 a^i5i—Liu(fiSilppi. aGtoi—dlumau utuesremp 
^ieu(pd(pifluj afiiumssT g^ji—pSiUsb ^jpdSlaSJ(f)ii)(3urTpi Qeu(piD^I ldSsIul/ 20 gtgst iDrrn$l loop-g> gsSJlI® 
QeuGtflQujpjSlppi. iSlasTasTtj s^© episode go (jpipuupptp QiDtrppii) arppastasT (Lpas)p parrppiehmpi 
<oT6STU6S)pU-ILb, <oTpp<S$)6$T (LfSG&ip UlUOSTOSMJpGUprTQST ^jl_pfl<si) ^jpdSl penalties GUrTtRlSllLIGfrGtTpi <oT6STU6S)pU-ILD 
QsiJGtflijuQppiSilppi. 


epochs, penalties, reward = 0, 0, 0 
while not reward == 20 : 

action = e. action space. sample() 
state, reward, done, info = e.step(action) 
if reward == -10: 

penalties += 1 
epochs += 1 


Timesteps taken: 1485 
Penalties incurred: 473 
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5. Q-Learning Algorithm: QLDfD3i<smi— uipuSlsb Qumppid 1485 (Lfists)/D r 5 atrjr 5 ap<sbsnss)j£U-iid, 473 
(LptsmjD psuforra ^j/oddl penalties eufTfhiSluqsrTsrrssiatUqid srrsmsorrid. g^uGurrap Q-Learning stssi i id 
RL-d&rrssTalgorithm-gr>u uiussiuGiiiapidGurTap, ^jidLO^IULiseh (^esirosnesi^d srrsssrsomd. s £0 
Agent (s,Lpriles)soudlsb a>ssrss)iss)i_uj Qaiusvidfs sj/dfouip Shesii—d&lesrtD Qsu^ld^J LD^luLfsesien 
rffes)issr<s^l(sc es)su£apd QsrressrQi ^q^esruLp ^Q&ujgvu®subsist (spsoid ^eurrps emend 
(3)<ss)fDdai QpLSf-iLj ld. ^apGurre&TfD memory-go Agent-d(&) si iiprits [ffd,#, algorithm 
LUUG$TU®£§j(5ijGg Q-table ^(ajid. ^Gilsb eyehQenrrq^ flesisod^iimesr usoGsurrp Qsiusbseiflesr 
^rrds risen ertf^ <sqeneqd(aj ^sqes)LDdlesrpesr srestnd ld^Iul/ Ga : L£ldsuu®dlfnap. £g)g/G>si/ Q 
LbShuLi ^/,0/i). £g)g/ spsdQsunrnj Q&ujgSIgst ^p^es)^iLjih 3-<smrj£gd a^ipiu^j. 

Q-table eresruap State * Action sremnd ^yessfl suipsu <sqesnmji3sb ^es)LC>iLjid. < 3 i$nsuap Glumggid 
2_<sb<sfT500 r£les)sosi^id states ^seyid, epenGeunq^ riles) so d^umesr 6 Glsiusbseiflesr 
LD^IUL/s^id columns-^t,s6qid ^sqsmujiLjid. ^j^esnpuues)i_udlsb^rTesr Q LD^IUL/seh 
GsLildsuuddlesrfoesr. epsdQsurTrnj d]les)sod(djid erd,^ Qsujsvidsrresr Q ld^Iul/ dlls urns 
2 _ehenGpn <sqspGsiJ QunqTjggumesT Q&iusonsd sessrddlsb QsrrehenuuQlid . 

6. rpid(Lpss) i _ iu flpsSlsb (LpgsSlsb 0 LD^luiSlemesru Quip ftp s£lsnri(ajid Q-table 

S-QTjeijnd&uuGldl/Dap. ^j^esr dimension 500 * 6 stgst ^(njdtsjihuip iSlesreuq^LDrrnp 
<sqes)LC>dsuu(i)d}/Dap. ££)uGunsp 328-susp fiemsodsrresr <sqss)ssrd,sq Glsiusosi^id 0 eremnd 
spends, LD^luipsemenu QurofTpehenfem^d snessrsomd. 

q table = np.zeros([e.observation_space.n, e. action space, nj) 

print (q_table[328J) 

[0. 0. 0. 0. 0. 0.J 

^jLDLD^luLisesierT update Q&iuu-iid rilsLpeurrssTap 100 (Lpesrfo for loop Qpsoid rfl&LpppuuQdltng]. 

&>it<° uap 100 (LpestfD Lnussrrjsss)end sfliunsd Qsrressrd) Gsrjdsu ums$Tu(ffS>luj Q LD^luipseh 
eyenQenrrq^ (Lpem/nuqid ^j^lsb update QsiuuJuuQiid. syehQsurrrn) Qsuj solid <sq&JD(aj (jpesresrij 
<sqd]ls Q LD^Iues)uu Qu/psp <sqsm rinfs, QaiusSlesruip 6 VLfilr 5 i—£&uu@)id. ^eheurTforrs 100 
episodes rfem/Dsp/oro iSlesresrrj rSqGtz flemsodsrresr Q ld^Iuli iSlesrencj^LDrrrrp ^qesns enemas 

SUIT (ofiftTGV fTLD. 

[-1.12496344 -1.12745359 -1.09896023 -1.0872184 -2.80419013 -2.78076376] 
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^jLDLD^ULi&iQnm osxsu&tgju umjd^LbGurrgj ^SdSlis, ld§>1usis)uu (-1.0872184 )Qujdq] 
aSlsnrhifgjih 3-sugj Q&iustnso g£ir5r&6S)6od(3ju Qurrq^^^LDrrssr Q&iusorr&i srQ^gjd 
Q^mshsTTecmh. ^ssmsv ^giGumsirfn (&j<ss)/D[5& episodes-go r£l&Lp^§)l syq^ (yuq_sqd(^ 
su pd^un^j. ^snpGiu learning by greedy STSsrGurriL. (3j<ss)/D[5& ulI&ld use sou&iL 
&LpfDd}&GS)smurT6ijaj] rglaLp£$£liu lSIsstsstGp srdGl&iusb Qurr([!)&,&,LDrrosT^rrs, ^6$)imLjih stsstjd 
(Lpt^siyd50 rsLDLimsb sup (LpLq^iLjLb. ^j^ssr lSIsstsstp &os)udl 100-sugj episode-sb mnp^luSlssr rs&psij 
srsmsssfldsm&iLi ld, surrtddliupenalty ld$£iul\ld QsusifluuQ^uuLLQGrrsrTgi. Gldsvild 
& rjrr&ifhuns, syqij episode-d(3) ^^ss)3uu LD^lLiLfaisb srssrQssrssrssr stsstu^ld 
Q susifluuQpguuLLQshsrT&ii. ^LbwSiluLiash {g£ipjD(3j (Lpssrssrrj QsusifluuQptiluj 
LD§>lULI<9i6S)SfT S)!lll—d (3jSS)JDSU /Td5 ^6S)L£>SU6S)$d 3>ITSSSTSOITLCi. {gfigjGsU petrpSil Q-algOrithm QpSOLC) 

dlrDuuna Q&iusbuud ajnrgjd Qarrsm® sfiJuusnp P_(£j§iluu($)£§iLb. 

Timesteps taken: 331 
Penalties incurred: 16 
Average timesteps per episode: 452.2 
Average penalties per episode 24.67 

7. sysuQsurrqij &Lprnd)u5lspul> Q-Table go Gloldu^^^lci rgJaLpsurrssrgi iSlssrsuq^iL &LDS$rurruLps$T 
u up fBssu _ (ol uiryLD. 

new value = (1 - alpha) * oldvalue + alpha * (reward + gamma * next max) 

^j^lsb alpha, gamma ^dhusnsu hyperparameters . Alpha srssrugj spsuQsurrq^ &Lp did) u5l solid 
uujssTuQ^uuQLf) learning rate-god (ajtfildt&jLb. £g)g/ 0.1 stsst <sq sv> ld pgl srrsrTgj. 

gamma stsstug)] 0 (Lpgsv 1 susmp ^ss)ld[b^ wdilustnuu Clujoj^q^d^Lb. £g)/5V0 0.6 stsst 
'S4S$)LDr5§lsrrsrT§i. £g).$65r LD&hu&nu 0 -<5@ QrBQijdap^sb ^ssuDd^idGun^], quick rewards-go 
ss)su^§i Qurrp^^^LDrrssr Q&iusmsop Gt5tjipQt5®d(&jLb. l-d(&) ^sqq^Slsb 

<gj6S)LC>d(3)Lt)GurTa)], long-term rewards-go ss)su£§i GprjrsGl&Qdt&jLb. 
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(old soil!) epsilon <oT6S)iLC> 6£0 parameter $jrzJ(3j over fitting-go fi gQfd&u uiuesru(fi)d)iD§l.for loop-sir 
gisudagdiltsb, $j[5& ld^Iuoduu QurrfgjgGg s^0 Q&ilussxso random-^ag G^tj^Q^(fi\d3i 
GsusmfifiLDrT<3i<sv<sv§] Q ld^iu&duu Qu/d/d «£0 Q&iussxsog G^tfriQ^ifiidsi GsusmQfLDrr 

<oTG5TUG$){5 (LpipSLj Q^IULL/LD. SjQsSTSsfleb SpsijQ(SlirT(T^ (Lp<o5)fDU-lLD SQ LD^IUSS)UU Qu/D/D 
Q&UJGV&Gm dfKnyLDud, GflQTjLtug GgrjrsQ&Qdau u (if) idGurTgi over fitting <5 j/bul_ sumuuL] 
^shm^]. ^aGsu gaSitjdai epsilon-d(&j s ^0 ld^iulSUssxsst sues)piu^j^^i 

^i^esns}.uuss)i_uS}sb Hfisc&LDiuLb random- <^,<ssiy ld, disc & ldujld Q ld^Iul/ Qarnsmi— 

Q&UJ6£JL£) G^ljl^Q^(fi)d3iUU(fi)d)fD§l. 
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19 (Lpuf-6y<ss)!J 


g )^/_sot Deep Learning (ipLsf.su<ss)i^r5&n sfili—oflsb.ssHso. AI, Neural Networks <s tgst^j 

UGcQgU^J Lj3jl<5m£)3i<SfT KGStiflssfl 2_6v£)<5V pt-figl SI/0® SOT/D SOT. ^<SUfD<SS)fD ^jGSXSSSTUJ^^lGC 

§?Uss)i_d(gjLb u it i—rhi a err, gugsksvuu^ Gdansk, StackOverFlow Gumssrr) pemhiasn, 
arTGlemmsiflaush sul^IQuj Qprri—ijrBgi «/D/p/ <surr G6u<sm®£lGrn<s£T. 


/5SOT7/5) 
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20 lSI/d L£)<ssr<ss)rT<sb&<srr 


http://freetamilebooks.com/authors/nithyaduraisamy/ 



srstfJuj iSiaJ^ila 

MfGLL 
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21 <9><5$sfl UJ LD UJDtfil 


$)<svd(gja<otr 

auli—rorD asssflijiiJ-u£$ 5 lssT srsifliu stfsfyiufhi&srT Qgm_riidl ^^}^iL_ULDmssr ^Lb&rhjash suss >/r 
tSyffilrB&ili— sS)ss)y)iLjLh srsuQ^d^ih QpssisuiurrssT gasusb&ssisrr Q^m^sjdffnurra,^ ^q^th &>sfTLDmu 

2_0©L/fl2/61/g/. 


• U, spsfl, spsifl STSST UsbsgJJl_& SUSSl&aSlflsgJ LD G)S}GUIJ[hl9i6S)GfT t5(njGU3)l. 

• $i£§]6S)fDu5)ssr rgJaLpsyasstsrr sr($)£§]ss>[juu§]. 

• STSUQjjLC) urhiasiflda sj^sumu lurrsuq^d^LDrrssr QrsnShfilsb s&su rrfhj&sstsn suLprhi^su^]. 

• suLSf-sfilsyLC), Lj^&rhi&Gfm&snih, suL^^ds.sfTrrs.sifd) aS\suprhi3iss)sn QsusiftuSlQsug]. 


urzi&eifldsi 


sfilqTjuuqpsrrsn srsuqjjLb uihjasifldGsomb. 

auli—jD/D «<S33 fljgiLiuLC) &rrrjr5& stfstyiuLtma £g)0<$<$<si) Qsusssr@Lb. 

uihj&sifldap ©^/T/_/570/i) (Lpssrssrrj&sssfhu^§>lfD(& ) 3_[hJ&(srTjss)i_uj u^uL/flw^ss)^ 
^siflda sr^rrurrrrd&uuQ&.tffifj&sh. 

editor@kaniyam.com (jpasajifl^aj efilsuijrEjaenLrEj^luj LnuQeomsjToorD 

2 _p^lQLDrTL^IujrTLij ^6ifl.$p6)!5lL_($l iurT0LD Q^m_rEj<ffia)rTLb. 
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• lm—gQgst Qurra^efr: utgiuLidunb ^e)fluq 

• ldi _ so s-errerri—d&ii) 


• STSSTSsmsi asssfluj£$5lfD<9;rT<a; ^ugi/ULiuL/ffliii uosM^ULfaish ^tssxssr^^iih 
a<sssfluj£$5liDairTai (LpgGsrqpgsomu usmi—dauLuh—grra 
^^j®iueiflddlGpeir. 

• $I£<s$tQ u rTQjjil® <oT<s$rddl(Ti)d<3;d&n.Lci-UJ u^uyiflLDg^GtnssT a<sssfluj££ 5 lfn(&j 
<SULprhl(3j&>lG(D6ST. 

• ^_/57«(ffl5/_(S3>uj (LpqguQuiurj, GgSil. 


prrrhi&Grr urdansiflda; g^q^ldlild s ^0 u(^®u5l<sv QsuQfDrTQ^sur} sjfDanssrGsu urdansiflgaji 

GUQTjSilfDrTlj <oTGsfl<S$T ^GUQTjlsiT £g)<S3>633775g/ UGSsftILIrTJDJD (LfiGS)GSTUJ<5^ LD. 

anl®<ss)[ra;<sh QLcrrLflQuuJijuLiamrTaiGijLC), oSls^,uJi£>rSl^ spq^surj Q&rrsbsod Gaul® 
aifDfru ^jiuiDiDUULli_ss)suujrrai^m ^(njdansvmh. 

ussu^uLjaush Glprri—ija surra sy lc> £g)0<s<s<sy/r/i>. 

QgrTLflsb rjjiiluLb, (olarrsnssia oS]sndaih, iSlrjd&rrrjLb, asmg, Gai<sS\d&\£Gjl!jih, 

(oTgstu usva^esxsuansifl^iLD ^j^^]ss)fod(^ QurrQ^rB^iLDUL^ujmssr ^dariiasurra 
^IQrjdansvmh. 

ff,!hjs,(<sr^d(ff ) g^uJsbumssT <oTr 5 pG)<surT([ij rFGtni—uS) spill) sTqggsoml. 

prhiansrrgi ussii—unassisn srsifhuQ^rTQ^ 2_®nrr ^susmujrra editor@kaniyam.COm 
(Lpajsurfldfg^ssriuiSlstDsudaspil. 

&><ofT urrmorfluLi, ^prjsusiflggsb 2-srrsrrl ili_ <5jss)ssruj oSl^ihiasifl&^il urhiaisifldaiscml. 

6Diurh]&<sfii(n)UiSi<s$T editor@kaniyam.com LDisShu/B/Bsyil. 
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eSlememuuthia&r 


• &<sssfl£ QgrriflsbnjiLlugGtnp ^ffluj s^hsmLpu^id LDdaqsrijd&rTa GwrnQarTGrrGrTuu®Lb 
(Lpiufodhurr^LD £g)g/. 

• $l&>)<sb udiasiflda ^rndsnsfr <s>j§ilijiLdu <^L,fDfD<sb sumurB^surrrrsi ^)0<s« G<5u<ss$r®Lb <oT<sstid 

.35/_!_/_/TIL/L/$)<SU<KO<SU. 

• & > !hl&,{6fl)d(& ) Qgflrsp G^ls^UJ^SS)^ £g) IU6STJD loTeifllU (LpGS)fDuSl(SC <oT(l)ld)3jl<o$)ud&) <^1TSULD 
^(ijjrB&rTsb Gungiih. 

• ^i^ssr (susnrfddl it, id spsijQsurrQ^suiflssT sidshiSIsviGld s_si kerr^]. 

• (§<mfD&<si?l<5fi(!]juiS}<ssr (Lp<s&)jD uj rT3> Qgiflujuu(b)£§il (Lp<ssTG<ssrfDfD£§>l/D(0j si///) <5U(3jda<sijLCi. 


QeueiflufLL® efteurjih 

udilULjflLmd © 2019 GGStifllULC). 

GGStifliupdilsb GhsusrrluSh—uuQLt) aLdQsmrr&m 

http://creativecommons.Org/licenses/by-sa/3.0/ ud&pgisv s^errsn difiGiui^eij aiTLCKssrerb 

QrBifl&GsxsnGhurTg&ii GULprhjauuQdUssrjDGsr. 


g^pdruisi-, 

asssflujpSiJsb (o)si/syf)si/ 0 /i> s.id^us^ijs.s^efT 3,ssisflujd,^liD(^Lb u<s$)i_d,&, ioT(L^^rT<sn(T^d(^ih a^fliu 
&msiriD(si?l£§j, rs&GleoQda, sfilrflGujrTdlda, usviD&rTiDiD, tqjjDJDLJ ls}~ < 3 l<sisuDd,§]d Glamsherr, 
Q^rrLfUsb GrBirddlsc u uj sir u ($&,&, j*j6$)ild& 1 GULprhjauuQdlfDg]. 

^diflujrf: <$. ^ssf)si7/T^<sOT-editor@kaniyam.com +91 98417 95468 

aulQGtnij&Gifleb Qsueifluu(^)^uu(^ili a(ri)£§jda;Gh aLLQetnijuJrTldJiflujQTjdGa a^flujssT 


151 



22 


GHSSsfllULD ^JD&&LLl—®n<olfT 



22.1 Q^rresxso ( 3/5 /tv® 0 - Vision 

gL&Lp Qumifl LLjDQjLb ^Gsrd(aj(L£>da<sh &rrrfr5& QLDiurfi&rrsijmrzi&m, aqijaSl&ish ldjd^jlci 
< 3tri5}6ii£G)prT(3j§ila<sh, ^ssxsstgu ([])&(&, ld &l1i_/djd ^6s$)i&&&,&) so Si issued 0zi> (^,Lpsc 

22.2 Lnsssfl {§j<sod(3j-Mission 

^nfloShusb LDjnrgjLb &QP&U Qumr^mrr^mj snerrijddld^ spuu, &>l£1lp Qu^rrifluSUssr umssrurr® 
sumrjsuss)^ s_q]^iuu(^i^^<sij§ilL, ^<s$xs$t£§i ^rSlsifd, Gumrd&^il) >5ll/d/d 

r$l6S5)ldad&)c5b ^>]6ti)65TG)J([ljd(& ) Lf) SlSS)l _ d&dGl&lU&tGVILD. 
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22.3 


pjoGumstopiu Q&ujevasb 


• &6ssfliuLC) L&GSTGsfl&iLp - kanivam.com 

• ®/f)(3ujL/y<su 3>rrLD<sir&T s-flesnriuSlsc £g) scsu& &>l£1l p L&tssTGinrTsbaGh- 
FreeTamilEbooks.com 

22.4 &lLi—JDJD QLDsirQufTQ^Llam 

• 2 _<s$)u <s$&Sl LDmofifl - Text to Speech 

• <oT(L£>4<sf,i<sssrffl - Optical Character Recognition 

• (a$?<S@PU3(5t)<gffi/<S<S/7W <oT(L£>43jl6mifl 

• i£}sstGS)rT<sb3i<sb SlsmLQ.sv amsfildfffj ^mspiuu-m&v - Send2Kindle 

• <s)SldSluiSLQ.ujrT(S)Shn3irTssr @/p/ gjonsfilgush 

• LSJsirsprTsbaiSfr^cm<surrd(^Lb 3j(rr,aS) 

• 3_<sg>/7 spsSl LDrnnnfl - (§)<ssxssstuj Q&iusSl 

• ^iscdShuLD- ^L<ssrL^irmu(d) Q&iucs6) 

• FreeTamilEbooks — nniuCh) Q&m<s61 

• FreeTamilEbooks — g»<gxoT<srl Q&iusiSI 

• WikisourceEbooksReport (j^ddhu QLDrTLfl&iissrijda&irTGST <stfIddi^Lp<svLb i£lssr^jT<sb^Grr 
udilGfilfDdau ulIl^ujgc 
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FreeTamilEbooks.com - Download counter LflssTs^sbaisrT uSjistffnd&u ulIi^ujso 


22.5 <j>/(5)<5<5 fglLLi—ihi&Grr/QiLesrQurTQjjiladr 


sfilddl (yxsopSilsb ^shsn LSlssrstnrrsbastftsn u^^lQrsij/(LPQ£> Grsrju usMfhunsnrj&sh Qpsoii> 
G^lsS)U[B§] lSl(SS)Lp §il(l]j£§]g6b 

(Lp(L£> Qrsrr rflrjsosmrj us$$fluJLDij£§>l uscQsu^j ^ ll / d/q Q ldsstQ u rrq^LdaisrT 
a_0Si//T<s0<5<su 


• ^lSIlp NLP danrssr uu5ltn&)u uLdismjnasrr r 5 i—£&n&sb 

• ansssfluJLb surr&aij sulLi—LD 2_(rijsii rr<d(&jgsb 

• sllidid QldsstQ urrqtjLdam, SlflQiuidL^sij am£>6si& S-fflsmLDuS) sc sumrii^onm 

^_0Siy/Td50L/Siy/T<95(S3>S)Td5 <S633Tl_fl5)/5g/ 2md(&jS)S){5t5SC 

• &SSsfllLlL£> ^j^Lflsb < 3 ]§> 1 < 9 ; udl&SfTlljurrSfTlj&SinGfT 2_0<S1//7<S0<5<SU, UuSljD £) ^sAggSC 


• LSlssrs^rrscndaidaiidr^ spqij ^ssxsmuj^sn 6 )&ujs6) 

• <oT(Lg£sji6m!fld(gj «£0 ^ss)smuj^sn Q&iusfl 

• ^LfilLp GpeSlGiurTstnisush a_0Siy/T«® QsusifhfilQpsb 

• OpenStreetMap.org sc asnsn l-ld, 0^0, Quujtj&s$)Gn pLfilLpnd&ih Gl&iupsb 

• pL&LprBrr® (Lp(Lgsuss)^iLiLb OpenStreetMap.org so susmrr^sb 

• (gLprBstngd &ss)pass)srr spsSl suuj-sfilsb suLpdu^^sb 


154 





Ta.wiktionary.org <g> sp(L£>rhi(pu(^p^l API d(p Gprrprra umrBrgjpsb 


• Ta.wiktionary.org d&n&i s^<sS\uu^i<s^ Q&iLULjih Q&iusfi s-(psurrd(ppsc 

• Pl£1l£ <oT(L£>pg]U L5l<SS)Lpp§il(lljplpl 2_061V/T<S50<$<SL) 

• pl£ 1 l£ GGurjdQ&rrsb amssniLb «0Si5? s-(psurrd(ppsc 

• <oT<sv<ovrT FreeTamilEbooks.com LSlssrs^ijsbs.s^smqii) Google Play Books, 
GoodReads.com sv < 5 jprnip<sv 

• ^l£1lP <S/D<35 £g )SSXSSSTUJ Q&lU<sS) 2_061//T<S0<5(SL) 

• pi£lip <oT(Lg^<s^LD uiq-dansyih &jdjd ^ssxsmuj Q&ujsSI s-(pGnrrd(ppsc ( 

aamozish.com/Course preface Qurrsv) 


Gm/Daemi— &>U-ii_rdaeh, QldsstQ urrq^LL&ssxsfT ^(psurrddl Q&iusvuQpp ^onssrsuflssr 

^pprrsyLf) Gpesxsu. 2_rd&mrT(sb <or<sij<sunGrn<ss)iih urdasiflda (j^ujgvilI) mstsflsb S-fd&m 

sfihsurrrd&smm kaniyamfoundation@gmail.com d(p lDqstsst^&gv ^oniuLidi&m. 


22.6 G) 6U 6lfl LJ U G8) L 4£3jG8T68)LD 

GessfliuLC) jd d&Lilies)srruSlesr Q&iLHsbash, jplLLi—ihi&Grr, QLLssrQunq^LLaisn lurrsi/Lh 

^g)/<s5)<s3rsi70<s0/i) Qurrpisurrssrprrsis^Li), 100% Q<su<sfiluu(SS)i^dp<ssT<ss)L£>iLjL^ss)iLb ^(pd(piL._ 
@ssxsssruiSlsb Q^iusbaissxsniLjLL, $)d& @ssxmruiSlsb ump ^rfydsms;, su/rsiy Q^susiy 
stisurrfii&idSfT, /_gn <ld amsmsornh. 
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GessihuLC) ^jodaiLii^issxsmiSlsc s_(j^Gimds,uuQud Q ldgstQ u sq^CsiGh lumsi^LCi < sl _/_/ d/d 

QldsstQ urr(rt)Lla<srTrr<9i ^pso rgl rrsyL—<sir, GNU GPL, Apache, BSD, MIT, Mozilla ^diu 
s_fl<ss)LD3i<oifl<sc GpGsrforrsi Q<suGiflu51 i_u u®id. 2 _(iT)Gij ndsiu u ® id lSIjd GUGnrdsiGh, 
qs5)«L/ULffii«6fr, g^gS) dGs.su LjSrGTT, ssQGSsrsGiflsGrr, i£\GSTGS)SGOSGh. sidQiGsiijsGfT ujsGi^id 

UJ rTGUQTjLf) udl(l])LD, U UJ GST U ® id GUGSISuSIgO Si fflGlLI L-iq-GlI SITLDGSTS ^^GSUDuSIgO (°£j([l)d(S)ld. 

22.7 r5<5srQamss)L- 

^riism s,gstQssgs)Usgtt s > L£l(Lgds>rrGST suu/d/d Gumsisssim ^(r^susd^id QsuJGVSGSim dlfops, 
gugsisuSIgv aS\GS)tjip§i Qsiuuj 2end(3jaS\d(3)Li). 

iSl GST GU (ITj id GUfhldd SSSSTsdlGV 2 _rhlSGn TBGStQsSGSU^SGSIGTT SWGSlIUlSl. 2 _I_GgST G)S}GUIJthl&GS)GTT 

kaniyamfoundation@gmail.com idlGsiGsrid^sGo s^GsuuLfsism. 


Kaniyam Foundation 

Account Number: 606 1010 100 502 79 

Union Bank Of India 
West Tambaram, Chennai 

IFSC - UBIN0560618 

Account Type : Current Account 


22.8 UPI Qeiuetia^darrm QR Code 
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(^t£luLf\ @so UPI QsiusSlssifleb £g)/5;5 QR Code Qguosxso Q&iluuttldsv Quits, so ml), ^ssldiuld 
QllQsc 2_mm surddld &Gmd(&j srsm, IFSC code ®o uiusstu^^si^ld. 


Note: Sometimes UPI does not work properly, in that case kindly use Account 
number and IFSC code for internet banking. 


UF 0 !* 


BHIM UPI Payments Accepted at 

Kaniyam Foundation 



Account Number: 606101010050279, IFSC Code: UBIN0560618 

Scan and Pay using any UPI supported Apps 




O 

PMTm 

O 

SBI 

T 

iMobile 

BHIM 

PhonePe 

PayTM 

SBI Pay 1 

Google Tez 

© 

e 

H 

m 

□ 


RBL Pay 

DBS 

PNB UPI 

Yes Pay 

AXIS Pay 

Chiller 

( 0 1 

V 



boi^t 

U 

Union UPI 

HDFC 

Baroda Pay 

Indus Pay 

BOIUPI 

Maha UPI 


Generated By: https://nsisodiya.github.io/Universal-UPl-QR-Code-Generator 
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